NLM objectives of Adult Education implies not merely 3Rs but also awareness and improvement as the outcomes expected of the programmes of literacy. Awareness, functionality and improvement. The Dave Committee signals the concern to capture the broader view of literacy as NLM objectives set forth and lay down a reliable instrument to measure literacy level reached. NLM norms of literacy is broader, as it includes functionality dimensions, and higher than Census definition and the framework for measuring learning outcomes is more reliable and rigorous. The Ghosh Committee viewed the reliability of the External Evaluations suspect, because the methodology, in terms of its technical and scientific soundness was also seen to suspect. About literacy norm, the Ghosh Committee felt that the level needed to be less and regarding awareness and functionality dimensions, it did not give either parameters/elements or the framework for measuring. The Ghosh Committee found many EEs have gone beyond mere literacy achievement testing and tried to capture functionality and impact dimensions. Framework for measuring other aspects[of a broadened vision of literacy and literacy level/competencies] can be abstracted from these evaluations to evolve (i) a definition of literacy norm; and (ii) framework and measurment tools. A lot of debate and concern about the reliability, technical soundness and scientific character of EE could be seen in the analysis of the Ghosh Committee. Also a lot of debate about the feasibility of "total literacy" at 80% level of success. Expert Group's strtong reservations about EE and wanted EEs to provide objective and reliable assessment of literacy and social impact. In the Expert Group's view, EEs are to be done by persons trained in social science research methodology and statiscians. EET's involvement in cuncurrent evluation by visits to TLC 2-3 times when TLC is in progress, for secondary data collection, observation and interaction. Mustaq's concern is how to evaluate learning outcomes of TLCs and not a multi-dimensional research with testing, interviews, secondary data, observations, group discussions. The Expert Group's concern was not with the problems about compromises in the rigour and fairness of testing or what all aspects to be measured and what design, framework etc. Its concern was mainly about unrepresentative, non-random and inadequate sampling design. The concern of the Expert Group for an evaluation that is representative, random and adequate in sample design must be appreciated: the LCs, 275 when the Expert Group deliberated (Aug. 1994), and ever growing in number (nearly 450 by Aug. 1997), implemented in districts as the basic unit, often with .2 to .4 million non-literates target size cannot lent itself for external evaluation of all its learners. But at the same time, a number of factors such as the high prestige TLC enjoyed, a high visibility, media attention, etc., led to `targetis' and the tendency to inflate signifcantly achievement figures. The pre- occuation with achievement of `total literacy', led to one-sided focus on quantitative aspects, with a pressure to report high percentage of achievements. The Expert Group viewed that External Evaluations should provide:

an objective and reliable assessment of literacy achievement and other social impact; feed back to local organisers about the present status, its strengths and weakenesses for remedial measures; acdemic inputs to improve policy and planing at District, State and Central levels. The approach should be participatory, involving participants of the programme and non-threatening. Otherwise, the pressure of targetitis of `total literacy' leads to maninpulation of achievement figures. EEs must adopt a multi-dimentional research approach, supplementing testing and interviews by secondary data, observations and group discussions.

The sampling design suggested for EE is based on the principle that every learner gets an equal chanvce of being included in the sample, while also covering the varying social demographic and regional characterstics of the district. Thus, the selected sample should be representative of the target population and one should be able to draw valid conclusions, based on the sample of learners tested, about the characterstics of the entire target population. Following this principle, 5-10% of the target learners by stratified random sampling is suggested as the size of the sample, besides a sub-stratum within the sample to represent the social disadvantaged groups, religious minorities and rural-urban variations. (+ how to select the 5-10% of the learners, ie. sample design) Village(ward in urban areas) would be primary unit of sample while external evaluation was considered opportune whenn 60% of the learners have completed the third primer, all learners in the sampled villages/wardsare to be covered under external evaluation. Based on the performance of learners in sampled villages/wards, the success rate at th e stratum level, i.e., Panchayat, Block, could be worked out, and for the entire district, by taking weighted average of success rates of different strata. At least 80% of the learners of the sampled villages/wards must be tested including home visits to catch up with absentee learners.