Developmental Evaluation Framework
Developmental Evaluation Framework
-
What is it?
A framework to track, assess and feedback on __change__ initiatives in complex or uncertain environments. It strongly involves _long-term and trusting_ relationships between evaluators and project or program staff. It is based on structured systems thinking and real-time [big-]data analytics (of the system in development or in a state of flux).
DE characteristics:
- needs a high level of interest/learning [in both the 'DE' framework and the 'object' system];
- needs for 'quality evaluators' in consultative roles to clarify assessments and to identify emergent (newly discovered) factors;
- may use traditional evaluation tools and techniques, and may also use traditional data systems;
DE applications:
It is often quoted that "in absence of a suitable evaluation framework", DE can provide the framework for the development of the (more specific framework for) evaluation system and 'the target system' (in complex, changing and partial environment). So, DE is used for 'two birds with one shot'. Because, DE is a long and involved process, reports (and assessments) for (successes of) parts of 'the projects' are lacking.
My questions on DE:
(Are there full case study examples? Internet searches fail to find one.)
- Is DE an optimizing technique/strategy?
- Does DE have exact demands? (One clear demand is for 'competent evaluators')
- Does the goal of using DE include training of DE practitioners and solving business problems?
My experience on a complex system:
I recalled working on 'optimization of a complex system' (in math language it's __to find least number of shortest lines to visit all pairs of points on a 2D plane__ in real life it's to find M taxis to service N calls for share-ride travel in a city --so that M is smallest and the distance travelled by M taxis is also shortest.). A lot of problems in management are like that -- to find the best 'actions' (plan) to achieve a goal with minimal costs. What I learned was that 'feasible set of solutions' can be very sensitive varying with 'conditions' and possible solutions have strong demands for accuracy (variables' values). As a consequence, solutions are __single use__ - not likely reusable. [Though, my computation of solutions was sufficiently fast (in 5 minutes with an old Pentium and 1GB RAM).] So, I learned something but what I made then was useless for repeating business operations.
What is a Framework? [added 5/12/2566]
In a simple way, let us draw a house. We will see ‘conceptual parts’ that we think a house should have: wall (body), door, windows, roof,… (parts vary with our perception of a house). That is a framework for a house. When we start building a real house, then we would have (architect/draftsman) blueprints (with more specific details) for (engineer) builders and craftsmen to install the various parts. There is a ‘control’ system to monitor, assess and feedback (all the way back to the ‘drawing’) to make sure that the house we build is ‘safe’, functional, on time, on spec and and on cost,…
So from one framework, we may see many ‘specific implementations’ (all with common conceptual parts but different designs and constructions of physical parts. All parts and the ‘whole’ should serve (some minimal) functional needs. There is then a ‘higher’ need to ensure that all buildings (construction and practices) meet ‘quality assurance’ (standards as tested by certain standard methods). Here we see that we have a richer field to work on and many people may use the opportunities for self-interest. (Sigh!)
More [paranoid] on frameworks: [Added 5/12/2566]
A common pattern in politics and intelligence is to ‘plant’ some people (agents) [with]in organizations of interest to learn and assess ‘changes’ [on and in] the organizations. DE framework specifically provides positions and opportunities (as ‘trusted evaluators’) that can be of security concerns. Other frameworks may offer similar opportunities in consultative or collaborative positions.
Can this pattern be used to improve outcome or combat corruption in government organizations? Frameworks that continuously monitor and assess operations and changes in organizations have potential to ensure ‘correctness’ (desirable outcome) from ‘go to woe’ (beginning to ‘fault’). [The current mode of operations for ‘investigative offices’ is to wait till faults occur then learn and fix. This reliance on ‘forensic’ evidence can reduce substantially amount of available ‘data’ and hence the just outcome.
What if continuous monitoring and assessing [framework] is automated in every government project? Will this reduce national corruption? Set a new channel for a different culture for servicing the public? We need more (paranoid) thinking to discover the frameworks used in corruption and to design counter-corruption frameworks?
ความเห็น (3)
Excuse me, Sir. Is DE came from capitalism or not, for i.e. socialism. There for should best fit for their background atmosphere. I am sorry for my stupid question, Sir. Thanks for the good note of this blog today.
I did not have any political inclinations in my mind when I wrote this blog. The urge to share my ideas on DE came as I thought about framework such as in the Common European Framework of Reference for languages (CEFR) by which we failed to gain our competency in English (ดัชนีความสามารถทางภาษาอังกฤษ https://www.gotoknow.org/posts/716504 ) and several posts by Prof. Vicharn Vanich (eg. Developmental Evaluation : 76. DE ของ ระบบ ววน. : ปัจจัยความสำเร็จ https://www.gotoknow.org/posts/716582 ) – these are tools for improvement.
What is a Framework? In a simple way, let us draw a house. We will ‘conceptual parts’ that we think a house should have: wall (body), door, windows, roof,… (parts vary with our perception of a house). That is a framework for a house. When we start building a real house, then we would have (architect/draftsman) blueprints (with more specific details) for (engineer) builders and craftsmen to install the various parts. There is a ‘control’ system to monitor, assess and feedback (all the way back to the ‘drawing’) to make sure that the house we build is ‘safe’, functional, on time, on spec and and on cost,…
All these considerations lie in all human activities in any country –universally.