Two theories on the test bench: Internal and external validity of the theories of Ronald Inglehart and Shalom Schwartz

https://doi.org/10.1016/j.ssresearch.2012.12.009Get rights and content

Abstract

In the last decades value research has produced a vast number of theoretical concepts. However, it is unclear how the different value theories relate to each other. This study makes a first step toward a systematic comparison of value theories. It focuses on the individual level of the two approaches that are, at present, probably the most prominent in international research – the theory of basic human values of Shalom Schwartz and the postmodernization theory of Ronald Inglehart. Using data from the World Value Survey and the European Social Survey for West Germany we assess both the internal and the external validity of the two accounts. The results indicate that both value theories have different strengths and weaknesses. Whereas the Inglehart account has lower internal and weaker construct validity, the Schwartz account is somewhat less consistent in its predications. Nevertheless, both value conceptions are able to explain a substantial share of variation in specific attitudes and behavior.

Highlights

► Comprehensive empirical assessment of Schwartz’ and Inglehart’s micro-level theories. ► Schwartz – Pro: Elaborated measurement model, acceptable internal validity in SEM. ► Contra: studies use different measurement models; (only) moderate external validity. ► Inglehart – Pro: Elaborated relationships with external variables. ► Contra: Low internal validity; the predictive power is sometimes overestimated.

Introduction

The empirical research of the last decades has produced an impressive number of different value orientations. Sometimes values are equated with more or less abstract, positively evaluated objects or states: Health, family, work, religion and many other entities are therefore called values. Sometimes values are related to basic human needs, like the needs for security, affiliation, or love. In the classical tradition values are defined as standards such as the values of freedom, equality, justice, or fairness. Apart from these principle disagreements about the concept of values, there are differences with regard to specific values. Two authors may use the same value name but understand and operationalize the underlying value differently or they assign different value names to very similar sets of indicators.

Different value researchers do not completely ignore each other but they quote the studies of others selectively and usually only in those cases where the findings of the other seem to support their own view.3 Comprehensive studies of the relationships between different value approaches are completely lacking. It is almost certain that problems of discriminant validity would arise if similar values from different theories were included in one and the same study (Jagodzinski, 2004). International comparative studies so far do not allow a comprehensive assessment of advanced value theories. It is true that the World Value Survey 2005 also includes 10 items of the Portrait Value Questionnaire of Schwartz in addition to the indicators of Inglehart’s value dimensions. However, it can already be anticipated that 10 items cannot adequately cover the 10 broadly defined value orientations of Shalom Schwartz, which is discussed in more detail in Sections 3.1 General problems and limitations, 3.2 Internal validity. Survey research may be reluctant to include the measurement instruments of different value theories into their questionnaires partly because they do not want to confront the respondents with batteries of similar questions and partly because it would increase the costs of such a survey immensely. Therefore, at the moment it cannot be said whether value research violates Occam’s principle and multiplies entities, in this case: values, beyond necessity. It is very likely that it does but no one can presently prove this.

In order to overcome the present situation, this paper attempts to systematically compare two very prominent value theories, the theory of basic human values of Shalom Schwartz and the postmodernization theory of Ronald Inglehart (e.g. 1977). Both authors present two-level theories, which distinguish between macro-level cultural values and individual-level value orientations. It is true, the focus of Inglehart’s (e.g. 1977) research has recently shifted to such an extent to the macro-level that the micro-level component of his theory can be overlooked. As the postmaterialism theory is only rudimentarily integrated into the new, more encompassing approach, one may gain the impression that we actually deal with two theories, a micro-level theory of postmaterialism and a macro-level theory of self-expression values. This is not the view of Ron Inglehart, however. Even his publications on macro-level cultural change persistently emphasize that cultural change is the result of micro-level value change (see, e.g., Inglehart, 1997, Norris and Inglehart, 2004, Inglehart and Welzel, 2005). The analysis of Inglehart and Baker (2000) further shows that cultural values and individual-level value orientations are operationalized with the same set of indicators. Due to space limitations, we have to confine ourselves exclusively to the key concepts of the individual-level value orientations in both approaches, which for the sake of brevity will be simply called values.

A comparison of two value theories should, first of all, investigate the internal validity of the measurement. Recent methodological studies on the measurement instruments of the ESS give important insights into this field, particularly also into problems of measurement equivalence, but they investigate only rudimentarily the predictive power of the underlying concepts. This is largely consistent with the strategy of Schwartz and his colleagues who mainly concentrated on the internal structure and validity of the values and only sparsely examined the relations between values and external variables. As long as this part of the theory remains less developed, however, the theory is of limited interest for the nonexperimental social sciences, which have always seen the main attraction of value theories in their promise to explain a broad range of beliefs, attitudes, and behaviors by a limited number of values. A comparison of value theories can, therefore, not be based on internal validity alone (Jagodzinski and Manabe, 2009, Opp and Wippler, 1990). Relationships with external variables, which either predict values or are predicted by values, are at least as important.

As both theories relate values to a set of common external variables, the strength and signs of these relationships will be the second criterion, which for the sake of brevity is called external validity of the theory. A theory is externally valid if all relationships have the theoretically predicted signs and the explained variance in all dependent variables is high.4 Though the predictive power of the values is in the focus of interest, the paper will also investigate the effect of selected exogenous variables on values.

Besides internal and external validity, the parsimony is used as a third standard of comparison. If two value theories have more or less the same explanatory power, the one with fewer values is more parsimonious and, therefore, superior to the other. So we have three criteria which we apply step by step to the two value theories. Before we do this, we very briefly discuss communalities and differences in the theories of Ronald Inglehart (1977) and Schwartz, 1992, Schwartz, 1994. The internal and external validity are examined in Section 3. As the study has to rely on two separate surveys, we use the European Social Survey (ESS) 2004 for measuring the values of Schwartz and the World Value Survey (WVS) 2005 for measuring the values of Inglehart. Needless to say, the external validity can only be assessed with regard to those external variables which are at least similarly measured in both surveys. Results are summarized and discussed in the last section.

Section snippets

The two value theories – Similarities and differences

Space limitations do not allow a comprehensive discussion of the two theories. The values of both theories will be very briefly described and compared in Section 2.1. The basic features of the measurement models are examined next (Section 2.2). The last Subsection discusses the relationship between values and a subset of external variables, which are similarly measured in ESS 2004 and WVS 2005. These relationships are summarized in a set of hypotheses (Section 2.3).

General problems and limitations

Before the analysis can be carried out, two important decisions have to be made. The first is favorable to Inglehart. Jagodzinski has carried out the same factor analysis as Inglehart and Baker (2000) with the data of the WVS 2005 and has shown that only the West German factor pattern was in line with the former results (see below, Section 3.2.1). Therefore, we confine our analysis to West Germany (N = 1851 in the ESS, N = 988 in the WVS).

Summary and conclusion

Theories of values and value change help us to understand differences between individuals and cultures. Yet social scientists have identified so many values during the last decades that the question arises of whether we really need them all. From this perspective, the two, presently, most prominent micro-level value theories have been investigated – the value theories proposed by Shalom Schwartz and Ronald Inglehart.

The World Value Survey 2005 includes all items for measuring the values put

Acknowledgments

We are indebted to Eldad Davidov and two anonymous reviewers for helpful comments. The work of Peter Schmidt was supported by the HSE Basic Research Program (International Laboratory for Sociocultural Research). We thank Lisa Trierweiler for proof reading and language editing the manuscript.

References (67)

  • A. Diamantopoulos et al.

    Advancing formative measurement models

    Journal of Business Research

    (2008)
  • S.H. Schwartz

    Universals in the content and structure of values: theoretical advances and empirical tests in 20 countries

    Advances in Experimental Social Psychology

    (1992)
  • Barnea, M., 2003. Personal Values and Party Orientations in Different Cultures. Doctoral Dissertation, The Hebrew...
  • T. Beckers et al.

    Congruence and performance of value concepts in social research

    Survey Research Methods

    (2012)
  • C. Beierlein et al.

    Testing the discriminant validity of Schwartz Portrait Value Questionnaire items – a replication and extension of Knoppen and Saris

    Survey Research Methods

    (2012)
  • M.D. Bezonsky et al.

    The how and what of identity formation: associations between identity styles and value orientations

    Personality and Individual Differences

    (2011)
  • W. Bilsky et al.

    Values and personality

    European Journal of Personality

    (1994)
  • K.A. Bollen

    Structural Equations with Latent Variables

    (1989)
  • B. Byrne et al.

    Testing for measurement and structural equivalence in large-scale cross-cultural studies: addressing the issue of nonequivalence

    International Journal of Testing

    (2010)
  • G.V. Caprara et al.

    Personality and politics: values, traits, and political choice

    Political Psychology

    (2006)
  • H.L. Costner et al.

    Deductions from axiomatic theory

    American Sociological Review

    (1964)
  • E. Davidov

    A cross-country and cross-time comparison of the human values measurements with the second round of the European Social Survey

    Survey Research Methods

    (2008)
  • E. Davidov

    Testing for comparability of human values across countries and time with the third round of the European Social Survey

    International Journal of Comparative Sociology

    (2010)
  • E. Davidov et al.

    Explaining attitudes towards immigration policies in European countries: the role of human values

    Journal of Ethnic and Migration Studies

    (2012)
  • E. Davidov et al.

    Bringing values back in: the adequacy of the European Social Survey to measure values in 20 countries

    Public Opinion Quarterly

    (2008)
  • E. Davidov et al.

    Values and support for immigration. A cross-country comparison

    European Sociological Review

    (2008)
  • N.D. De Graaf et al.

    Intragenerational stability of postmaterialism in Germany, The Netherlands and the United States

    European Sociological Review

    (1989)
  • B. Duriez et al.

    Authoritarianism and social dominance in Western and Eastern Europe: the importance of the sociopolitical context and of political interest and involvement

    Political Psychology

    (2005)
  • A.H. Eagly et al.

    The Psychology of Attitudes

    (1993)
  • European Social Survey Round 2 Data, 2004. Data File Edition 3.2, 2011. Norwegian Social Science Data Services, Norway...
  • M. Fishbein et al.

    Predicting and Changing Behavior: The Reasoned Action Approach

    (2010)
  • S. Flanagan

    Changing values in advanced industrial societies revisited: towards a resolution of the values debate

    American Political Science Review

    (1987)
  • N.D. Glen

    Aging and conservatism

    Annuals of the American Academy of Political and Social Science

    (1974)
  • G. Hofstede

    Culture’s Consequences. Comparing Values, Behaviors, Institutions, and Organizations across Nations

    (2001)
  • R. Inglehart

    The Silent Revolution: Changing Values and Political Styles among Western Publics

    (1977)
  • R. Inglehart

    Post-materialism in an environment of insecurity

    The American Political Science Review

    (1981)
  • R. Inglehart

    Aggregate stability and individual-level flux in mass belief systems: the level of analysis paradox

    The American Political Science Review

    (1985)
  • R. Inglehart

    Culture Shift in Advanced Industrial Society

    (1990)
  • R. Inglehart

    Modernization and Postmodernization. Cultural, Economic, and Political Change in 43 Societies

    (1997)
  • R. Inglehart et al.

    Modernization, cultural change, and the persistence of traditional values

    American Sociological Review

    (2000)
  • R. Inglehart et al.

    Modernization, Cultural Change, and Democracy. The Human Development Sequence

    (2005)
  • W. Jagodzinski

    Materialism in Japan reconsidered: toward a synthesis of generational and life-cycle explanations

    American Political Science Review

    (1983)
  • Jagodzinski, W., 2004. Methodological problems of value research. In: Vinken, H., Soeter, J., Ester, P. (Eds.),...
  • 1

    Fax: +49 221 47694199.

    2

    Fax: +49 641 9923059.

    View full text