| Dr. Richard Kohr , Department of Education, Bureau of Testing and
Measurement is cross examined by Mr. Schmidt. Dr. Kohr has been involved in all of the
statewide tests over the past 20 years EQA, TELLS and PSSA. EQA was a test that was to
measure the effects of the consolidation of school districts in 1965 to see whether
"Bigger is Better." It was terminated with the beginning of the TELLS test which
was a broader measure of skills than was the EQA. TELLS purpose was the remediation of
individual students. During the time of TELLS the Secretary of Education changed and made
the test much more public and comparative. The test served as an early warning to school
districts in grades 3,5 and 8 about the students. It only gave a partial view of the
school because of the cut off scores. There was a peer review of the questions and the
cutoff scores were decided. It only decided about minimum competency vs. Borderline. It
showed the youngsters that were candidates for remediation. TELLS was phased out during a
time when remedial money was not longer available. Dr. Kohr denied that the notion of
cheating that he described the previous day had anything to do with the demise of the
test. Dr. Kohr was shown a document produced by his bureau or division showing
correlations between wealth factors and success on the TELLS test. The relationships were
significant in a statistical way. There was also a description of outliers and a formula
that you can use to determine if an object ( or in this case school district) is an
outlier.
PSSA comes in conjunction with the strategic plans that are done with the districts.
Its complete use by all school districts is only since 1995. There were a number of
questions about NAEP and Pennsylvania's participation in it. The conclusion reached by Dr.
Kohr was that it interfered with the giving of the PSSA. He did not see the utility of
using this national test for comparisons with other states. Are certain tests more
diagnostic or prescriptive than other tests. Yes they are such as the Metropolitan, Iowas
and such. If students needed more of a diagnostic or prescriptive test would they use the
aforementioned. The answer was yes.
Within the documents used to compare the testing with poverty, why was free and reduced
lunch used instead of AFDC (which is used in almost all the funding components over the
years. Since there is a falling off of kids in the high school, poorer districts might be
disadvantaged by the use of such a figure.
The state has used the terms rural urban and suburban in their determination of the
similar school profile. These are means of self identification. The other identifier is
free and reduced lunch to determine poverty. According to the school profile, one would
find outputs of similar schools to be similar. Are there other things that might affect
outputs. The answer in the school profile booklet is leadership, policies and resources.
Does this mean that there is a relationship between resources and outputs? Advanced
placement courses appear to be a significant indicator of a quality educational program.
In the profile it is stated that "all students may not be going to college."
The question then arises who determines whether a student goes to college. Dr. Kohr
prepared, or was asked to prepare certain documents related to expense data. There were
many questions relating to the meaning of what the mean of all expenses was. Was it a mean
of mean or an actual mean of all expenses in the state divided by the ADMs. There seemed
to be a difference in an earlier document prepared by Barbara Nelson who used a different
mean ( or average). There was a distinction made between instructional expense mean and
total expense mean and what figures were used for what years. Dr. Kohr was asked to look
at the York City School District to see if it was the same number on both documents and a
REX report.
There was also discussion about the document produced by Dr. Kohr and some conclusions.
It appeared on the tables presented that the scores on PSSA actually went up as expenses
went up and as expenses went down scores went down. There were "outliers" such
as Juniata County. The question was asked if raw data was available to Dr. Kohr. Would the
aggregation of raw data given us a more precise set of answers?
On redirect by the defense Dr. Kohr was asked about the variability in expense in all
the groupings that he had calculated. There appeared not to be such a great variability.
There were further questions about the standard deviations in each group. Dr. Kohr could
not answer that question from the tables. The defense pointed out in the TELLS correlate
paper that there was a statement that Tuition, which may be a proxy for expenses was not
necessarily correlated with results on the TELLS test. It was not as significant as market
value per pupil, AFDC or other variables. A correlation does not imply causation. They are
only associations. There are other things that can be used such as a multiple regression
analysis. . Leadership and policies are also significant.
Mr. Schmidt asked if the tables in Dr. Kohr's presentation were absolute means or
averages of all the means. He said that they were the latter.
Mr. Steve Simchock is in the Bureau of Curriculum, Evaluation and Reports and is the
coordinator in the production of the School Profile. He is involved in the collection of
the data from two sources for the profile- the Department of Education's data file and
surveys sent to the schools. He described the makeup of the profile from the regulations
of the State Board of Education and said that the Secretary of Education could add or
subtract categories. He reviewed all of the categories and described whether they were
from the surveys or the department's data base. Expenditures per students were divided in
1/5ths and noted in the profile.
In cross examination the questions related to the use of expenses per enrollment rather
than actual ADMs. That was a choice in the accumulation of data. The question of the
Special Education Childcount related to enrollment and ADMs. If there were questions about
the data, the districts were called to confirm the answer. If an answer could not be given
then no data was entered.
Mr. Simchock was asked if these seemed to be inconsistent. He did not think that he
could answer the question.
Dr. William Fairley is the state's expert witness. He described his credentials with a
B.A. From Swarthmore, an M.A. From Harvard and a PHD from Harvard. He did some tangential
work on the Coleman Report. He did work with Mostetler on the Quality of Educational
Opportunity which was a review of the Coleman Report at the Kennedy School of Government
at Harvard. He has taught at NYU, Harvard for six years in public policy, worked for the
Commissioner of Insurance in Massachusetts. He has also taught courses at Temple,
Swarthmore and his been a visiting professor at NYU and the University of Karachi
(Pakistan). While he worked for the Commissioner of Insurance he developed a theory of
modern financial economics for the insurance industry that would allow the state to set
rates. . He formed his company "Analysis and Inference" in 1979. His principal
areas of work are finance, discrimination, insurance, education, auditing, risk analysis
and welfare program quality. He did serve as a school finance expert in the Massachusetts
equity suit but it never got to court. He was involved in the stipulations. The state lost
the case in Dr. Fairley's words. He has written many articles for journals in finance,
economics and insurance.
In presenting Dr. Fairley the court and the plaintiffs were asked to accept Dr. Fairley
as an expert. Upon this question, Dr. Fairley was asked if he had any association with any
school finance organizations. He answered no. Was he a reader of such journals. He
answered no. Was he familiar with Mosteler's recent work on class size and achievement in
Tennessee. He did not know. The plaintiffs asked that he be admitted as an expert witness
only on statistics and economics. The court allowed that.
On direct examination , Dr. Fairley was asked where he had gotten all the data for this
study. He said from the Department of Education, the census, from Dr. William Cooley of
the University of Pittsburgh. He also got national data on the SATs. He got national
school statistics from the Department of Education in Washington.. The last group of
information was gathered from standard sources.
Property per capita was used in descriptions on the first table rather than property
per student. The districts were then ranked by deciles ( 50 districts of various sizes in
each decile). "You can dream up questions about why either could be used," It
does not make a great deal of difference." "Not muddied by the object of the
resources.( the students)" Why did you use property value per person (per capita). It
is the resources in the community. Personal income per person is the same as market value
per person.
The medians are used in each of the deciles in what is called a box plot which shows
outliers and the range within the decile. This is a more descriptive demonstration of how
things are. The median is a better measure of central tendency than other measures in this
case. Skewness detracts from the description of the middle values.
In other charts, other than box plots, higher market value school districts are
providing more local revenue the corollary is also true. There is an association ( a
correlation) between spending per pupil in the box plots and the property deciles. That is
insignificant in the first 5 deciles more pronounced in the 6th and 7th and very true in
the 87, 9 and 10th. However, when you invoke the cost of living , explained as a national
way of equalizing values of houses between urban and rural areas there is much less of a
correlation in total and only in the last two deciles is there a significant difference.
In this case spending was tabulated per pupil. This still does not relate to causation.
Within the correlations, Judge Pellegrini asked what a .60 correlation meant. Was it
significant or not. Was it above average. The answer was yes. However, that does not apply
causation.
A perfect correlation is 1.0 an absolute nothing correlation is 0. The median is a
better measure and it shows that there is an overlapping of districts in expenditures in
all the deciles. When rearranged for cost of living ( housing only), the bottom line in
all the deciles appears to be the same, excluding outliers.
Dr. Fairley was asked if in the course of his report he had looked at whether
"spending affects educational results." He said that he had read 5 major sources
and was not convinced that it does. The five sources were : Hanushek, Chubb and Moe, the
Coleman Report, Hedges and Burghes (editor). Hedges criticism of Hanushek was not
convincing. He did not really do an exhaustive literature search for "Does Money
Matter"
He did an original analysis of the TELLS data with relations to actual instructional
spending. He found the average of the TELLS tests in all three grades (six scores) in the
90-91 year. He then compared it to the spending . He did not adjust the spending for cost
of living. "If you avoid adjusting , do it" The correlation is .50 for all the
districts. He said that he did not conclude anything from this because he could not
determine causation.
He described a parable of a situation where one could attribute a relationship of large
feet to good spelling. That is a good association . However, the causation is not there
until you realize that older children have bigger feet. That would describe causation. He
said that there was a way of getting out that information by something called multiple
regression analysis.
If you can keep some variables constant, you can exclude them from causation. This
method can then isolate the probable cause. These relationships between variables can be
described by a multiple regression correlation. In doing these analysis he found that the
socio-economic relationships to be much stronger than the spending ( which showed little
in the way of statistical strength).He used TELLS tests of previous years as a variable.
He also did an analysis of the current PSSA test using as a variable the TELLS test of
6 years before. He did not figure in transience and assumed that the bulk of the children
were the same.
He also found that socio economic variables were statistically significant in this
multiple regression analysis.
"Did you address the question of Equity in ;your Report?" No I did not answer
that question. I am neither a Rabbi or a Priest. That is a value judgement. "
Alexander and Salmon aggregated the data differently. They , as the blind men and the
elephant looked at different parts of the elephant. I believe that I saw more of the
elephant.
Mr. Schmidt than cross examined Dr. Fairley. Mr. Schmidt noted that Dr. Fairley had
been deposed and that Drs. Alexander and Salmon had not. Mr. Schmidt asked how many pupils
there were in Pennsylvania. Dr. Fairley said that he had mispoken when he said 360,000
during his deposition. He was then asked where he got the census data for the number of
people in the school district. He was not clear, but he thought it might be the Department
of Education. He was then asked if he had used the 1980 or 1990 census data. "It
doesn't change that much." He had said in his deposition that he wasn't sure which he
had used. He was now confident that it was 1990.
What year did Dr. Fairley use for the Market Value . He said the 1993-94 data. In any
part of your report did you use assessed value. He said that he had not. Mr. Schmidt
pointed to a part of the deposition where he had used the term assessed value. He was then
asked about STEB. He said it was an organization in the state that equalizes market value.
He was then asked to identify the market value of the districts on the REX report. He
was also asked if the used the Market Value and Personal income per WADM that appeared on
the REX report. He said that he had made his own calculations. He did say that he checked
a few. He was asked the term Total spending meant with relations to expenditures per WADM
in Departments calculations.
He was asked if the had run calculations on market value per pupil rather than market
value per capita .He said that he did not remember. It was pointed out to him that in his
deposition he had said that he had done that and that the numbers had been different. He
was asked if he had included those charts and reports in his study. He said that he hadn't
Cross Examination will conclude of Dr. Fairley tomorrow
Dr. Eugene Hacek will testify. |
