POLITICAL SCIENCE 138 QUANTITATIVE REASONING IN POLITICAL SCIENCE

 

Internal and External Validity:

Bivariate Analysis.

Multiple Regression

Tables.


Internal and External Validity:

Links:

Expecially study the examples of internal validity in the Psych 404 page.:

Scenarios:

Internal and External Validity.

Identify the problems of Internal or external validity in each of these cases.  Explain just how the problems would affect the results.

 

Scenario 1:

After years of grading his Introductory American Government exams, Professor Casper Curmudgeon discovered a pattern: Students who received low grades (C or less) on his midterm exams consistently improved their scores – by an average of half a letter grade – on their final exams.  At first he thought this was due to sample mortality (weaker students dropping out of the course), but the effect remained when he calculated the averages for only for those students who took both tests.  He also discovered that students who received good grades (B+ or better) on the midterm had their grades drop half a letter grade on the final.

"Grades," Curmudgeon concluded, "are powerful motivating force."  Students who received good grades on the midterm, he reasoned, slack off in their studies while those who receive low grades try harder.  From then on he never gave a grade higher than a B on his midterms.

Scenario 2:

In 1980 the National Highway Transportation Safety Administration conducted an experiment to evaluate the effectiveness of a new high-mounted rear window brake light on passenger cars.  Working with a national rental car agency, they randomly installed rear brake lights on half the agency's fleet of cars.   At the end of a two-year trial it was discovered that the cars with the new lights experienced 35% fewer rear-end crashes and 25% fewer fatal accidents than the cars without the lights.   If installed on all automobiles, they reasoned, thousands of lives would be saved.  Subsequently, all automobile manufacturers were required to install the lights. 

But after 5 years, when 90% of all automobiles had the lights, the traffic fatality and rear –end collision rates had dropped only 2% (per million miles traveled).  What went wrong?

Scenario 3:

The Normal Police department instituted an intersection safety program.   At the beginning of the month, the department identifies the intersections with the most traffic accidents and then implements an intensive police patrol at those intersections.   The new patrol has been an enormous success, said the police chief, and has resulted in an average reduction of 35% in accidents at those intersections.
Does the program really work?

Scenario 4:

 Cutting taxes is not the way to economic prosperity.  The 1980 tax reductions instituted under President Reagan led to an immediate recession, rising crime and poverty rates and huge budget deficits.  The 1992 tax increases passed under the Clinton administration were followed by eight years of economic growth, steady reductions in crime and poverty, and the first budget surpluses in decades. 

Scenario 5:
In 1998 states that had instituted and used the death penalty had an average murder rate of 6.6%, while in the states that did not employ the death penalty, the murder rate was only 3.7%.

Clearly this shows that the death penalty, rather than deterring crime actually causes more of it.


Bivariate Analysis.

Bivariate Statistics

Pearson’s correlation coefficient

The strength and direction of the LINEAR association between two interval (not categorical) variables.

Values:   -1     to     0     to      +1

0 indicates no association

R-square

The strength and (and not direction) of the linear association.

In Multiple Regression (MR); the percent of the variation in the dependent variable that is explained by the independent variables.

Values:  0 to 1

 

 

 

 

 

 

Slope, unstandardized regression coefficient

The magnitude of the relationship between two variables.  Estimated increase in Y for each one-unit increase in X.  Positive or negative (if r is positive or negative).  

Used in the equation:      

Values:  from negative to positive infinity.

"
ß" in Caiazza and Ethridge readings

p value

 

 

 

Level of significance of a coefficient (r, R-square, b or beta)

The probability of getting that high a coefficient by chance (or due to sample error).  

Values:  .00   to   1.00   

Low values (p <.05) are significant,  the higher the r values, the lower the p value.

   

 

Multiple Regression:

Multiple Regression Terms.
(Dependent Variable: NAEP Reading Scores)

 

Predictor

b (unstand-ardized)

Beta
(standardized)

Stand.
error

t

p

 

SL:  Books

.4459

.2680

.2151

2.07

.046

*

SL: Software

.08659

.1929

.05403

1.59

.120

 

SL: Service

-1.2819

-.4566

.3073

-4.17

.0000

*

PL

1.3196

.2928

.5576

2.37

.024

*

Exp

-.000046

.0264

.0006309

-.07

.942

 

R-square = .602

 

N = 41 states; ap<.01; bp<.05

NOTE: RC = scores on NAEP Reading Comprehension Test, fourth-graders;
SL: Books = school library, number of books per child;
SL: Software = school library, software available;
SL Service = school library; services available;
PL = public library, annual circulation per capita;
Exp = expenditures per child.

Constant (or intercept) The estimated value of the dependent variable when all the independent variable equal zero. Generally, this is not a meaningful number.  Example: not shown in this table.

  The unstandardized regression coefficient (what the slope would be if all the other variables in the equation were held constant).  The change in the dependent variable for each one-unit increase in the independent variable.  Example: If the number of school library books per child were to increase by 10 (say from 50 to 60 books per child), we would expect the reading scores to go up 4.46 points, all other things being equal.

Standard error:  The confidence interval for the unstandardized coefficient.  Multiply this by 2 to get the 95% interval.  If the b is negative and is twice as large as its standard error, we can be 95% sure that the actual effect is indeed negative.  Example:  The effect of SL: Software is .087 plus or minus .108 (2 times .054).  Because the confidence interval ranges from a -.02 to +.195 we cannot be sure that effect of software is positive.  The coefficient (.087) is thus not significant)

t (or t-statistic or t-ratio).  The unstandardized coefficient divided by the standard error.  The unstandardized coefficient is regarded as significant if it is twice the standard error, if the t is larger than 2.

Example: SL books; SL service; and PL all have t statistics greater than 2.

 

p  The probability that the coefficient (either a standardized or unstandardized coefficient or a correlation coefficient) is due to the small number of cases.  A p value less than .05 is generally regarded as significant.  SL: Software and school expenditures have no significant effect on reading scores.

 

beta (standardardized regression coefficient).  What the correlation coefficient would be if all the variables in the equation were held constant. Compare this with the correlation coefficient to see if the original relationship was spurious.  Example: SL Servicee has the strongest independent effect on readings scores; the effect is negative; all other things being equal, states with more library services have lower reading scores.

R-square.   A measure of the accuracy of the whole equation.  The percent of the variation in the dependent variable that is explained by the independent variables.  Example:  The five variables in the equation explain 60% of the variation in readings scores.


 

Tables.

 

 

 

 

 

 

IMR

OWR

SEE

HE

ECC

GNP

Infant Mortality Rate

1

 

 

 

 

 

Organic Water Pollution

.41

1

 

 

 

 

Secondary Education Enrollment

-.79

-.20

1

 

 

 

Health Expenditures per capita

-.60

.17

.45

1

 

 

Export Commodity Concentration

.33

.44

-.25

-.06

1

 

Gross National Product per capita

-.80

-.24

.66

.73

-.20

1


*52 non-OECD countries
Burns, Thomas J, Jeffrey D Kentor, and Andrew K Jorgenson. 2003. "Trade Dependence, Pollution, and Infant Mortality in Less Developed Countries." In Crises and Resistance in the 21st Century World-System, ed. Wilma A Dunaway. Westport, CT: Praeger

 

 


 

 

 

1997
murder rate

gun laws

death penalty

unemployment

poverty rates

%Black

1997 murder rate

1.00

 

 

 

 

 

gun law strength

-0.09

1.00

 

 

 

 

death penalty
(1=DP;0=NoDP)

0.39*

-0.16

1.00

 

 

 

Unemployment rate

0.49*

-0.17

0.06

1.00

 

 

poverty rate

0.44*

-0.22

0.19

0.55

1.00

 

% Black

0.73*

0.04

0.30

0.20

0.27

1.00

per capita income

-0.24

0.65

-0.26

-0.33

-0.54

-0.05

*50 state data

 

 

 

8th grade Math

School Size

Two Parent Families

Positive Attitude

 6hrs.+ TV per day

8th grade NAEP Math scores

1.00

 

 

 

 

School Size

-0.53*

1.00

 

 

 

% students from
Two Parent Families

0.83*

-0.65

1.00

 

 

% students with Positive Attitude Towards Math

-0.28

0.18

-0.49

1.00

 

% students watching TV 6hrs.+ per day

-0.82*

0.50

-0.83

0.51

1.00

Equity (1=unequal; 0=equal)

0.22

-0.33

0.06

0.07

-0.15

*36 state data

 

 

 

 

 

 

 

SL: Books

SL: Software

SL: Service

PL

Exp.

RC

.495a

.377b

-.513a

.559a

.058

SL: Books

 

.423a

-.035

.453a

.205

SL: Software

 

 

.041

.315b

.044

SL: Service

 

 

 

-.188

.051

PL

 

 

 

 

.090

N = 41 states; ap<.01; bp<.05

NOTE: RC = scores on NAEP Reading Comprehension Test, fourth-graders;
SL: Books = school library, number of books per child;
SL: Software = school library, software available;
SL Service = school library; services available;
PL = public library, annual circulation per capita;
Exp = expenditures per child.

Stephen D. Krashen, "School Libraries, Public Libraries, and the NAEP Reading Scores" SLMQ Volume 23, Number 4, Summer 1995  (article on-line)
 

N= 245 cases (convicted defendants)
Dependent variable = Sentence Severity {ranges from 0 (for a deferred sentence) to 30 (15 or more years}.
(from Krashen, op.cit.):

Table 3. Multiple-Regression Analysis
(Dependent Variable: NAEP Reading Scores)

Predictor

b (unstandardized)

Beta
(standardized)

Stand.
error

t

p

 

SL:  Books

.4459

.2680

.2151

2.07

.046

*

SL: Software

.08659

.1929

.05403

1.59

.120

 

SL: Service

-1.2819

-.4566

.3073

-4.17

.0000

*

PL

1.3196

.2928

.5576

2.37

.024

*

Exp

-.000046

.0264

.0006309

-.07

.942

 

R-square = .602

 

 

 

Table 1: State 8-Grade Math Scores: Correlations and Betas

 

 

 

 

r

beta

 

 

 

Positive Attitudes

-0.34

0.21

 

 

 

Percent in Poverty

-0.75

-0.26

 

 

 

Pupil/Teacher Ratio

-0.20

-0.10

 

 

 

Teacher Salary

0.50

0.02

 

 

 

2 parents at home

0.84

0.49

 

 

 

6 hours + TV

-0.83

-0.43

 

 

 

    R-square

 

0.90

 

 

 

 

Table 2:
8th Grade Math Scores: Unstandardized Regression Results

 

 

 

Coeffi-
cients

Standard Error

t Stat

P-value

 

Intercept

208

29.4319

7.0846

0.000

 

Positive Attitudes

0.35

0.1700

2.00

0.048

 

Percent in Poverty

-0.47

0.1250

-3.7600

0.001

 

Pupil/Teacher Ratio

-0.46

0.2500

-1.8400

0.072

 

Teacher Salary

0.000010

0.000100

0.1000

0.404

 

2 parents at home

0.81

0.2500

3.2400

0.003

 

6 hours + TV

-0.82

0.2000

-4.100

0.000

 

    R-square

0.90

 

 

 

 

 

The following results are based on a survey of 350 radio 

newspeople working at US commercial radio stations.
_________________________________________
Table 3. Predictors of Radio News Salaries
_________________________________________
                            Beta       r
Market size (smaller)      -.61**   -.61**
Years in news               .42**    .57**
News staff size             .35**    .60**
Years at station           -.10*     .46**
Position                   -.08*     .07
Education level             .05      .06
Gender (being female)      -.01     -.10
Race (being minority)       .00      .02
Jobs held                   .02      .17**
Age                         .00      .40**
_________________________________________
N = 350  ** p<.001, * p<.05 and 
R-square = .70

_________________________________________



(see Vernon Stone radio and TV articles on-line)
_________________________________________


 

Table 1. Predictors of TV News Salaries 
________________________________________
                           Beta      r
Years in news              .54**    .54**
Position                   .37**    .38**
News staff size            .30**    .40**
Age                        .22**    .52**
Market rank (smaller)     -.21**   -.39**
Gender (being female)     -.07     -.15**
Race (being minority)      .05      .04
Job satisfaction (lower)  -.04     -.15**
Education level            .02      .06*
Jobs held                 -.02      .28**
Years at station          -.00      .37**
 
N = 1,550 (TV newspeople)
 
Betas (regression coefficients) 1-5 are significant (p<.001); 6-11, 
nonsignificant (p>.05).
 
Correlations are significant at p<.01 except Education (p<.05) & Race 
(nonsignificant at p>.05).
_________________________________________
 

The effects of City and Metropolitan Area Size on Local Civic Participation (Contacting Officials), with controls for Individual and Contextual Population Characteristics.

Source: Oliver, 2000 (Ethridge)
N=2,0032 (individuals surveyed)