Two theories on the test bench: Internal and external validity of the theories of Ronald Inglehart and Shalom Schwartz
Highlights
► Comprehensive empirical assessment of Schwartz’ and Inglehart’s micro-level theories. ► Schwartz – Pro: Elaborated measurement model, acceptable internal validity in SEM. ► Contra: studies use different measurement models; (only) moderate external validity. ► Inglehart – Pro: Elaborated relationships with external variables. ► Contra: Low internal validity; the predictive power is sometimes overestimated.
Introduction
The empirical research of the last decades has produced an impressive number of different value orientations. Sometimes values are equated with more or less abstract, positively evaluated objects or states: Health, family, work, religion and many other entities are therefore called values. Sometimes values are related to basic human needs, like the needs for security, affiliation, or love. In the classical tradition values are defined as standards such as the values of freedom, equality, justice, or fairness. Apart from these principle disagreements about the concept of values, there are differences with regard to specific values. Two authors may use the same value name but understand and operationalize the underlying value differently or they assign different value names to very similar sets of indicators.
Different value researchers do not completely ignore each other but they quote the studies of others selectively and usually only in those cases where the findings of the other seem to support their own view.3 Comprehensive studies of the relationships between different value approaches are completely lacking. It is almost certain that problems of discriminant validity would arise if similar values from different theories were included in one and the same study (Jagodzinski, 2004). International comparative studies so far do not allow a comprehensive assessment of advanced value theories. It is true that the World Value Survey 2005 also includes 10 items of the Portrait Value Questionnaire of Schwartz in addition to the indicators of Inglehart’s value dimensions. However, it can already be anticipated that 10 items cannot adequately cover the 10 broadly defined value orientations of Shalom Schwartz, which is discussed in more detail in Sections 3.1 General problems and limitations, 3.2 Internal validity. Survey research may be reluctant to include the measurement instruments of different value theories into their questionnaires partly because they do not want to confront the respondents with batteries of similar questions and partly because it would increase the costs of such a survey immensely. Therefore, at the moment it cannot be said whether value research violates Occam’s principle and multiplies entities, in this case: values, beyond necessity. It is very likely that it does but no one can presently prove this.
In order to overcome the present situation, this paper attempts to systematically compare two very prominent value theories, the theory of basic human values of Shalom Schwartz and the postmodernization theory of Ronald Inglehart (e.g. 1977). Both authors present two-level theories, which distinguish between macro-level cultural values and individual-level value orientations. It is true, the focus of Inglehart’s (e.g. 1977) research has recently shifted to such an extent to the macro-level that the micro-level component of his theory can be overlooked. As the postmaterialism theory is only rudimentarily integrated into the new, more encompassing approach, one may gain the impression that we actually deal with two theories, a micro-level theory of postmaterialism and a macro-level theory of self-expression values. This is not the view of Ron Inglehart, however. Even his publications on macro-level cultural change persistently emphasize that cultural change is the result of micro-level value change (see, e.g., Inglehart, 1997, Norris and Inglehart, 2004, Inglehart and Welzel, 2005). The analysis of Inglehart and Baker (2000) further shows that cultural values and individual-level value orientations are operationalized with the same set of indicators. Due to space limitations, we have to confine ourselves exclusively to the key concepts of the individual-level value orientations in both approaches, which for the sake of brevity will be simply called values.
A comparison of two value theories should, first of all, investigate the internal validity of the measurement. Recent methodological studies on the measurement instruments of the ESS give important insights into this field, particularly also into problems of measurement equivalence, but they investigate only rudimentarily the predictive power of the underlying concepts. This is largely consistent with the strategy of Schwartz and his colleagues who mainly concentrated on the internal structure and validity of the values and only sparsely examined the relations between values and external variables. As long as this part of the theory remains less developed, however, the theory is of limited interest for the nonexperimental social sciences, which have always seen the main attraction of value theories in their promise to explain a broad range of beliefs, attitudes, and behaviors by a limited number of values. A comparison of value theories can, therefore, not be based on internal validity alone (Jagodzinski and Manabe, 2009, Opp and Wippler, 1990). Relationships with external variables, which either predict values or are predicted by values, are at least as important.
As both theories relate values to a set of common external variables, the strength and signs of these relationships will be the second criterion, which for the sake of brevity is called external validity of the theory. A theory is externally valid if all relationships have the theoretically predicted signs and the explained variance in all dependent variables is high.4 Though the predictive power of the values is in the focus of interest, the paper will also investigate the effect of selected exogenous variables on values.
Besides internal and external validity, the parsimony is used as a third standard of comparison. If two value theories have more or less the same explanatory power, the one with fewer values is more parsimonious and, therefore, superior to the other. So we have three criteria which we apply step by step to the two value theories. Before we do this, we very briefly discuss communalities and differences in the theories of Ronald Inglehart (1977) and Schwartz, 1992, Schwartz, 1994. The internal and external validity are examined in Section 3. As the study has to rely on two separate surveys, we use the European Social Survey (ESS) 2004 for measuring the values of Schwartz and the World Value Survey (WVS) 2005 for measuring the values of Inglehart. Needless to say, the external validity can only be assessed with regard to those external variables which are at least similarly measured in both surveys. Results are summarized and discussed in the last section.
Section snippets
The two value theories – Similarities and differences
Space limitations do not allow a comprehensive discussion of the two theories. The values of both theories will be very briefly described and compared in Section 2.1. The basic features of the measurement models are examined next (Section 2.2). The last Subsection discusses the relationship between values and a subset of external variables, which are similarly measured in ESS 2004 and WVS 2005. These relationships are summarized in a set of hypotheses (Section 2.3).
General problems and limitations
Before the analysis can be carried out, two important decisions have to be made. The first is favorable to Inglehart. Jagodzinski has carried out the same factor analysis as Inglehart and Baker (2000) with the data of the WVS 2005 and has shown that only the West German factor pattern was in line with the former results (see below, Section 3.2.1). Therefore, we confine our analysis to West Germany (N = 1851 in the ESS, N = 988 in the WVS).
Summary and conclusion
Theories of values and value change help us to understand differences between individuals and cultures. Yet social scientists have identified so many values during the last decades that the question arises of whether we really need them all. From this perspective, the two, presently, most prominent micro-level value theories have been investigated – the value theories proposed by Shalom Schwartz and Ronald Inglehart.
The World Value Survey 2005 includes all items for measuring the values put
Acknowledgments
We are indebted to Eldad Davidov and two anonymous reviewers for helpful comments. The work of Peter Schmidt was supported by the HSE Basic Research Program (International Laboratory for Sociocultural Research). We thank Lisa Trierweiler for proof reading and language editing the manuscript.
References (67)
- et al.
Advancing formative measurement models
Journal of Business Research
(2008) Universals in the content and structure of values: theoretical advances and empirical tests in 20 countries
Advances in Experimental Social Psychology
(1992)- Barnea, M., 2003. Personal Values and Party Orientations in Different Cultures. Doctoral Dissertation, The Hebrew...
- et al.
Congruence and performance of value concepts in social research
Survey Research Methods
(2012) - et al.
Testing the discriminant validity of Schwartz Portrait Value Questionnaire items – a replication and extension of Knoppen and Saris
Survey Research Methods
(2012) - et al.
The how and what of identity formation: associations between identity styles and value orientations
Personality and Individual Differences
(2011) - et al.
Values and personality
European Journal of Personality
(1994) Structural Equations with Latent Variables
(1989)- et al.
Testing for measurement and structural equivalence in large-scale cross-cultural studies: addressing the issue of nonequivalence
International Journal of Testing
(2010) - et al.
Personality and politics: values, traits, and political choice
Political Psychology
(2006)
Deductions from axiomatic theory
American Sociological Review
A cross-country and cross-time comparison of the human values measurements with the second round of the European Social Survey
Survey Research Methods
Testing for comparability of human values across countries and time with the third round of the European Social Survey
International Journal of Comparative Sociology
Explaining attitudes towards immigration policies in European countries: the role of human values
Journal of Ethnic and Migration Studies
Bringing values back in: the adequacy of the European Social Survey to measure values in 20 countries
Public Opinion Quarterly
Values and support for immigration. A cross-country comparison
European Sociological Review
Intragenerational stability of postmaterialism in Germany, The Netherlands and the United States
European Sociological Review
Authoritarianism and social dominance in Western and Eastern Europe: the importance of the sociopolitical context and of political interest and involvement
Political Psychology
The Psychology of Attitudes
Predicting and Changing Behavior: The Reasoned Action Approach
Changing values in advanced industrial societies revisited: towards a resolution of the values debate
American Political Science Review
Aging and conservatism
Annuals of the American Academy of Political and Social Science
Culture’s Consequences. Comparing Values, Behaviors, Institutions, and Organizations across Nations
The Silent Revolution: Changing Values and Political Styles among Western Publics
Post-materialism in an environment of insecurity
The American Political Science Review
Aggregate stability and individual-level flux in mass belief systems: the level of analysis paradox
The American Political Science Review
Culture Shift in Advanced Industrial Society
Modernization and Postmodernization. Cultural, Economic, and Political Change in 43 Societies
Modernization, cultural change, and the persistence of traditional values
American Sociological Review
Modernization, Cultural Change, and Democracy. The Human Development Sequence
Materialism in Japan reconsidered: toward a synthesis of generational and life-cycle explanations
American Political Science Review
Cited by (68)
The pre-political origins and policy consequences of environmental justice concern
2022, Politics and the Life SciencesThe idea of a theory of values and the metaphor of value-landscapes
2024, Humanities and Social Sciences CommunicationsSuccessful Life Conduct in Very Old Age: Theoretical Implications and Empirical Support from a Population-Based Study
2024, Applied Research in Quality of Life