Which type of replication repeats the same research methods to see if the same findings are obtained?

Authors [Adams2005] define the following replication types: Literal Replications. Which are those studies where both measures and manipulations are as identical as possible to a previous study. Efforts are made to duplicate both the dependent and independent variables as closely as possible. Operational Replications. Which are those studies that alter the operationalization of the criterion variable while maintaining the use of treatments. It is desirable to retain the conceptual integrity of the measure and the independent variables are unaltered. Instrumental Replications. The treatments or independent variables are changed and the measures remain the same as in prior studies. Constructive Replications. Which vary both the criterion and the manipulations or treatments. An attempt to replicate conceptually a previous study is made.

Table of Contents Show

Bahr et al.
Barker and Gurman
Brown and Coney
Easley et al.
Evanschitzky and Armstrong
Hyman and Wright
Van IJzendoorn
Kantowitz et al.
Kelly et al.
Leone and Schultz
Lindsay and Ehrenberg
Mittelstaedt and Zorn
Tsang and Kwan

Bahr et al.

Types A..P. This classification [Bahr1983] categorizes replications according to four dichotomic properties (equal or different) of a replication. These properties are: time, place, subjects and methods. Based on combinations of these properties, Bahr et al. define 16 replication types.

Barker and Gurman

Authors [Barker1972] adopt the convention of viewing an experiment as consisting of two sets of variables; methodological and procedural matters on the one hand and outcome or dependent variables on the other. Within this framework the experimenter may seek to replicate a reference experiment with respect to: 1) both the procedural and dependent variables, 2) only the dependent variables, 3) only the procedural variables, 4) neither the procedural nor dependent variables. Procedural variables replicated; dependent variables replicated. The experimenter seeks to replicate both the procedural and outcome variables of a reference experiment. An attempt is made to duplicate exctly the sampling procedure, experimental conditions, measuring techniques and methods of analysis of the reference experiment.

Procedural variables varied; dependent variables replicated. In this type of replication, the experimenter seeks deliberately to vary the methodology and procedures of a reference experiment but aims also to replicate the results of the reference experiment. He tests he generality of some recognized lawful relationship. Procedural variables replicated; dependent variables not replicated.

In this type of replication, the experimenter does not use the same dependent variables as in the reference experiment. Procedural variables not replicated; dependent variables not replicated. In this type of replication, the experimenter deliberately varies the methods and procedures used in the reference experiment. Such a replication and the reference experiment are equivalent in terms of the concept of the variables used in the two experiments.

Beck

Author [Beck1994] groups different replication classifications into one ordering, which includes the following replication types. Type 1. Identical or literal studies are considered exact replications. Type 2. Virtual, operational, direct, and retest are terms used to indicate that a study is closely replicated. This strategies aproximate the original research design. Type 3. Systematic extension, constructive, systematic, and independent replications include the extension of an earlier study. This strategies increase empirical generalization by significantly modifying the original design. Type 4. Pseudo-replication and Internal replication, in this strategies data from both the original and the replication studies are collected at the same time. Type 5. Theoretical replication, this strategy develops and verifies a theory.

Blomquist

There are two types of replication Blomquist1986: literal and construct. In a literal replication, the researcher uses the same measures with the same type of subjects, and controls the same conditions. The original study is replicated as exactly as possible. In a construct replication, a second investigation begins with a similar hypothesis but uses new methods of measurement and design. The findings support the construct under investigation and can make the findings more generalizable.

Brown and Coney

Authors define the following two replication types [Brown1976]: Replication. A substantial duplication of a previously published, data based research project. Substantial means that any changes made deal with increasing the internal validity of the research effort (i.e. any change affecting the controls of a design). Replication with extension. A duplication of a previously published, data based research project with changes made in either the active (manipulated) or attribute (measured) variables, the intent of which is to increase the external validity of the research effort (i.e. the generalizability).

Easley et al.

Authors [Easley2000] describe four replication types: Type 0 (Precise Duplication). This replication is defined as a precise duplication of a prior study. Therefore, Type 0 (precise duplication) studies are those studies in which every nuance of the experimental setting is precisely reproduced; as such, the cause-effect relationship is finite. The ability to conduct a Type 0 replication is limited to experimenters in only some of the natural sciences. As others have stated, it is an impossibility to conduct a Type 0 replication in a social science context because uncontrolled extraneous factors have the potential to interact with the various components in an experimental setting. For example, human subjects cannot be precisely duplicated. A social scientist is limited only to matching subjects as closely as possible. Type I (Duplication). A type I replication is a faithful duplication of a prior study and, as such, is considered the "purest" form of replication research in the social sciences. It should be mentioned at this point that a Type I replication is the one most closely associated with the term "replication" in the minds of most researches. More over, this is also the type of replication research most criticized for not being creative. This is somewhat ironic, given the apparent receptivity of reviewers to cross-cultural research that, in many cases, is usually the study of the generalizability of findings from a single country or culture to others and, thus, is simply a Type I replication. Type II (Similar). A type II replication is a close replication is a close replication of a prior study, and a Type III replication is a deliberate modification of a prior study. Type II replications are the most common form of replication research in marketing settings and are useful in testing phenomena in multiple contexts. If effects are shown in a variety of testing contexts, the case for the findings is strengthened. This has been called the process of triangulation. Type III (Modification). This replication is a deliberate modification of a prior study. In a Type III replication, the threat of extraneous factors inherent to the nature of human subjects, unless explicitly accounted for in theory testing, is not a factor of concern with regard to replicability.

Evanschitzky and Armstrong

Where authors define [Evanschitzky2010]: Real Replications. This replication is a duplication of a previously published empirical study that is concerned with assessing whether similar findings can be obtained upon repeating the study. This definition covers what are variously referred to as "exact", "straight" or "direct" replications. Such works duplicate as closely as possible the research design used in the original study by employing the same variable definitions, settings, measurement instruments, analytical techniques, and so on. Model Comparisons. This replication is an application of a previously published statistical analysis that is concerned with assessing whether a superior goodness-of-fit can be obtained, comparing the original statistical model with at least one other statistical model. Data Re-analyses. This replication can be defined as an application of previously published data that is concerned with assessing whether similar findings can be obtained using a different methodology with the same data or a sub-sample of the data.

Finifter

Where author defines [Finifter1972]: Virtual Replication. The intention is to repeat an original study not identically but "for all practical purposes" to see whether its results hold up against chance and artifact. Virtual replications are also frequently conducted to find out how dependent a result is on the specific research conditions and procedures used in an original study. To answer this question, one or more of the initial methodological conditions is intentionally altered. For example, a survey or experiment might be repeated except for a change in measuring devices, in the samples used, or in research personnel. If the initial result reappears despite changes, faith in the original finding mounts. Systematic Replication. The emphasis in systematic replication is not on reproducing either the methods or the substance of a previous study. Instead, the objective is to produce new findings (using whatever methods) which are expected by logical implication to follow from the original study being replicated. When such an implication is actually confirmed by systematic replication, confidence is enhanced not only in the initial finding that prompted the replication but also both in the derived finding and in whatever theoretical superstructure was used to generate the confirmed inference. Pseudoreplication. It can be defined according to three main operational variations: the repetition of a study on certain subsets of an available total body of real data; the repetition of areal data study on artificial data sets which are intended to simulate the real data; and the repeated generation of completely artificial data sets according to an experimental prescription.

Finifter

Found in [Finifter1975] which it is the same clasification as in [Finifter1972].

Fuess

Fuess [Fuess1996] mentions several classifications, basically describing two forms of replication. Replication or duplication aims to run an experiment that resembles the reference experiment as exactly as possible, whereas replication with extension varies the replication to increase external validity (generalizability).

Hendrick

Where author [Hendrick1991] describes the following replications types: Strict Replication. An exact, or strict, replication is one in which independent variables (treatments) are duplicated as exactly as possible. That is, the physical procedures are reinstituted as closely as possible. It is implicitly assumed that contextual variables are either the same as in the original experiment, or are irrelevant. Partial Replication. A partial replication is some change (deletion or addition) in part of the procedural variables, while other parts are duplicated as in the original experiment. Usually some aspect of the procedures is considered "unessential", or some small addition is made to expedite data collection. Conceptual Replication. A conceptual replication is an attempt to convey the same crucial structure of information in the independent variables to subjects, but by a radical transformation of the procedural variables. In addition, specific task variables are often necessarily changed as well.

Hunter

This author [Hunter2001] defines the following replications: Statistical Replication. For statistical replications as perfectly replicated studies: -All studies measure the independent variable in exactly the same way. -All studies measure the dependent variable in exactly the same way. -All studies use exactly the same procedure. -All studies draw samples from the same population. Scientific Replication. For scientific replications for simple causal studies: -All studies measure the same independent variable X. -All studies measure the same dependent variable Y. -All studies use essentially the same procedure. -All studies should sample from populations that are equivalent in terms of the study question and hence the study outcome. The difference is that statistical replications assume that the word "same" means identical, while scientists interpret the word "same" to mean equivalent. Conceptual Replication. This replication verifies one of the hypotheses that were not tested in the original study. The researcher of the original study defines control groups to test the most obvious alternative hypotheses against administrative details that are thought to be irrelevant. Any treatment, intervention or manipulation is a set of administrative procedures, which are mostly intrinsic to the active ingredient of the treatment. These replications examine whether the administrative procedures influence the treatments as reflected in the dependent variable.

Hyman and Wright

Authors [Hyman1967] distinguish three kinds of research operations that are subsumed under the label "replication". -There are studies that build upon, extend, and ocassionally test under varyng circumstances the findings and hypotheses of previous research. -There are studies that attempt to duplicate, as closely as possible, the design, problem, hypotheses, and methods of earlier studies. -There is an attempt to conduct a study in several stages, extending it over a relatively long period of time in order to see what new phenomena emerge.

Van IJzendoorn

Where author [IJzendoorn1994] defines: Complete Secondary Analysis. It is a kind of replication in which all parameters except the researcher and the method of data analysis are kept constant. Secondary analysis also is one of the most inexpensive and efficient types of replication, because it is based on existing data sets. One of the main barriers to secondary replication is, however, the accessibility of the original data sets. The complete secondary analysis may include recoding of the original raw data. In this replication, there are two phases of processing the raw data involved: the coding and analyzing of the data. Restricted Secondary Analysis. In this type, the coding system is not changed but only the methods of analyzing the data, to see whether the original results survive statistical criticism or the application of refined methods of statistical analysis. Exact Replication. A replication will be called "exact" if it is essentially similar to the original study. This replication is applied to (dis)confirm the doubts, and to check the assumptions of the varied replications. Many scientists feel that exact replications may be carried out, but usually are irrelevant for scientific progress. Varied Replication. Replications should be carried out in which new data under different conditions are being collected. From the start, the original study will be "trusted" so much that rather significant variations in the design will be applied. Larger variations may lead to more interesting discoveries in addition to the original study, but they will be followed by smaller variations if more global replications fail to produce new "facts". If even modest variations fail to reproduce the results, a more or less exact replication is needed.

Kantowitz et al.

Authors [Kantowitz1984] describe the following types: Direct Replication. This is the attempt to repeat the experiment as closely as is practical, with as few changes as possible in the original method. Systematic Replication. The experimenter attempts to vary factors believed to be irrelevant to the experimental outcome. If the phenomenon is not illusory, it will survive these changes. If the effect disappears, then the researcher has discovered important boundary conditions on the phenomenon being studied. Conceptual Replication. One attempts to replicate a phenomenon, but in a way radically different from the original experiment.

Kelly et al.

Where authors [Kelly1979] describe: Literal Replication. The earlier findings may be reexamined using the same manipulations (independent variables, experimental procedures, etc.) and measures (dependent variables, methods of data analysis, etc.). Operational Replication. If the experimenter wishes to vary criterion measures, the experiment would be termed an operational replication. In this instance, the dependent variable would represent a different operationalization of the construct; the essential conceptual meaning would remain unchanged. Instrumental Replication. This replication is carried out when the dependent measures are replicated and the experimental manipulations are varied. Variations in the implementation of experimental procedures which do not go beyond the originally established relationship would be included in this category. Constructive Replication. A constructive replication attempt may be identified when both manipulations and measures are varied. This replication involves the attempt to achieve equivalent results using an entirely original methods recipe.

La Sorte

Author [LaSorte1972] distinguishes the following types: Retest Replication. In its general form retest replication is a repeat of an original study with few if any significant changes in the research design. The retest has two major purposes: 1) it acts as a reliability check of the original study, and 2) inconsistencies and errors in procedure and analysis can be uncovered in the repeat. Although the retest increases one's confidence that the findings are not artifactual, it does not eliminate the possibility of error in process, especially when the same investigator conducts both studies. Internal Replication. The differences between the retest and internal replication are mainly procedural. Instead of seeking confirmation of an original study, the internal replication is built into the original study design. So the data, part of which are used for the replication, are gathered simultaneously by the same investigator using a common set of research operations. One finds variations in the procedures for selecting the samples. Two of these procedures are: 1) drawing two or more independent samples, and 2) taking a single sample and later dividing it into subsamples for purposes of analysis and comparison. The internal replication provides an additional data supply which acts to cross-check the reliability of the observed relationships. Thus it is methodologically superior to the single study where the hypothesis is tested only once by one body of data. Independent Replication. Independent replication is the basic procedure for verifying an empirical generalization. It does this by introducing significant modifications into the original research design in order to answer questions about the empirical generalization that go beyond those of reliability and confirmation. The essential modifications include independent samples drawn from related or different universes by different investigators. These replications differ in design and purpose. They can, however, be broadly categorized into three problem areas. First, is the empirical generalization valid? Second, does further investigation extend it to other social situations or subgroups outside the scope of the original study? Or, third, is the empirical generalization limited by the conditions of particular social situations or specific subgroups? Theoretical Replication. It involves the inductive process of examining the feasibility of fitting empirical findings into a general theoretical framework. These replications seek to verify theoretical generalizations. In these replications, empirical variables, which have concrete anchoring points are abstracted and conceptualized to a higher theoretical plane, it is necessary to sample a variety of groups using different indicators of the same concepts.

Leone and Schultz

Where authors [Leone1980] define: Experimental Replication. The same experiment is conduced more than once, although there can be (especially with social systems) no perfect replications. It involves the same method and the same situation. Nonexperimental Replication. The same method is applied to different situations. Corroboration. It involves different method and same situation, or different method and different situation.

Lindsay and Ehrenberg

Where Authors [Lindsay1993] describe: Close Replication. This replication attempts to keep almost all the know conditions of the study much the same or at least very similar (for example, the population or populations in question, the sampling procedure, the measuring techniques, the background conditions, and the methods of analysis). A close replication is particularly suitable early in a program of research to establish quickly and relatively easily and cheaply whether a new result can be repeated at all. Differentiated Replication. It involves deliberate, or at least known, variations in fairly major aspects of the conditions of the study. The aim is to extend the range of conditions under which the result still holds. Exploring a result with deliberate variations in the conditions of observation is the essence of generalization. According to the authors, there are three reasons for running a differentiated replication: -Use different methods (different measuring instruments, analysis procedures, experimental setups, and/or investigators) to reach the same result (triangulation), -Extended the scope of the results, -Define the conditions under which the generalization no longer holds.

Lüdtke

Lüdtke [Ludtke2008] identifies two approaches to replication in the clinical field: exact replication and independent replication. Exact replication involves using the same dose of the same medicine for the same length of time and the same type of disease as in the original trial. On the other hand, a replication is independent when it is performed independently of the reference study. Lüdtke [Ludtke2008] gives two examples of clinical trials to illustrate each type of replication. Although the study object of the two trials is the same, they are completely different. The researchers of the two trials have different medical specialities, the research was funded by different sponsors and the studies were run at different hospitals.

Lykken

This author [Lykken1968] defines the following replication types: Literal Replication. This involves exact duplication of the first investigator's sampling procedure, experimental conditions, measuring techniques, and methods of analysis. Operational Replication. One strives to duplicate exactly just the sampling and experimental procedures given in the first author's report. The purpose of operational replication is to test whether the investigator's "experimental recipe" the conditions and procedures he considered salient enough to be listed in the "Methods" section of his report will in other hands produce the results that he obtained. Constructive Replication. One deliberately avoids imitation of the first author's methods. To obtain an ideal constructive replication, one would provide a competent investigator with nothing more than a clear statement of the empirical "fact" which the first author would claim to have established.

Mittelstaedt and Zorn

Where authors [Mittelstaedt1984] describe: Type I. The replicating researcher uses the same data sources, models, proxy variables and statistical methods as the original researcher. Type II. The replicating researcher uses the same data sources, but employs different models, proxy variables and/or statistical methods. Type III. The replicating researcher uses the same models, proxy variables and statistical methods, but applies them to different data than those used by the original researcher. Type IV. In this replication, different models, proxy variables and statistical methods are applied to different data.

Monroe

Where author [Monroe1992] defines: Simultaneous Replication. Does the same researcher in the same study investigate consumer reactions to more than one product, or to more than one advertisement? Sequential Replication. Does the researcher or another researcher repeat the study using the same or different stimuli at another point in time? Nonindependent Replication. The replication is conducted by the same researcher. Independent Replication. The replication is conducted by different researcher. Assumed Replication. For example, a researcher using both males and females simultaneously in a study and finding no gender covariate effect assumes replication across gender. Demonstrated Replication. What is preferable is separate gender conditions wherein the effect has or has not been obtained separately for males an females, that is, demonstrated. Strict Replication. The replication is a faithful reproduction of the original study. Partial Replication. The replication is a faithful reproduction of some aspects of the original study. Conceptual Replication. The replication uses a similar conceptual structure but incorporates changes in procedures and independent variables.

Radder

This author [Radder1992] distinguishes the following types: Reproducibility of the material realization of an experiment. In this type of reproduction, the replicator correctly performs all the experimental actions following instructions given by the experimenter who ran the previous experiment. This reproduction is based on a division of labour, where other previously instructed people can run the replication without being acquainted with the theory underlying the experiment. As in this reproduction it is possible to follow the same procedure to verify the outcome without detailed knowledge of the theory, there may be differences in the theoretical interpretations of the experiment. Reproducibility of an experiment under a fixed theoretical interpretation. This reproduction implies that the conditions of the previous experiment can be intentionally altered in the replications, provided that the variations are irrelevant to the theoretical interpretation of the experiment. Reproducibility of the results of an experiment. This type of reproduction refers to when it is possible to achieve the same result as a previous experiment using different methods. This category excludes a reproduction of the same material operationalization.

Schmidt

Where author [Schmidt2009] describes: Direct Replication. This involves repeating the procedure of a previous experiment. In this replication, the context variables, the dependent variable or subject selection are open to modification. Conceptual Replication. This is the use of different methods to retest the hypothesis or result of a previous experiment.

Sidman

In this classification author [Sidman1960] describes the following types: Direct Replication. This type of replication of a reference experiment is run by the same experimenter. The replication is repeated under the same conditions as the reference experiment and can be run using a between-subjects or within-subjects factors, leading to different single-subject design types. Systematic Replication. Instead of simply repeating the experiment, data gathered from the original experiment are used as the baseline for running new experiments and thus gathering additional related data. While repeating the same experiment, this replication changes some application condition, that is, the context, one of the subject characteristics, the experimenter, the problem behaviour, etc. A series of successful systematic replications that modify one or more of these factors provides more information about the generality of the outcomes.

Tsang and Kwan

This authors [Tsang1999] define the following replication types: Checking of Analysis. In this type of replication, the researcher employs exactly the same procedures used in a past study to analyze the latter's data set. Its purpose is to check whether investigators of the original study have committed any errors in the process of analyzing the data. Reanalysis of Data. Unlike the checking of analysis, in this type of replication, the researcher uses different procedures to reanalyze the data of a previous study. The aim is to assess whether and how the results are affected by problems of definition, as well as by the particular techniques of analysis. Quite often the replication involves using more powerful statistical thecniques that were not available when the original study was conducted. Exact Replication. This is the case where a previous study is repeated on the same population by using basically the same procedures. The objective is to keep the contingent conditions as similar as possible to those of the previous study. The researcher usually uses a different sample of the subjects. The main purpose is to assess whether the findings of a past study are reproducible. Conceptual Extension. A conceptual extension involves employing procedures different from those of the original study and drawing a sample from the same population. The differences may lie in the way of measuring constructs, structuring the relationships among constructs, analyzing data, and so forth. In spite of these differences, the replication is based on the same theory as the original study. The findings may lead to a revision of the theory. Empirical Generalization. In this replication, a previous study is repeated on different populations. The researcher runs an empirical generalization to test the extent to which the study results can be generalized to other populations. It follows the original experimental procedures as closely as possible. Generalization and Extension. The researcher employs different research procedures and draws a sample from a different population of subjects. The more imprecise the replication, the greater the benefit to the external validity of the original finding, if its results support the finding. However, if the result fail to support the original finding, it is difficult to tell whether that lack of support stems from the instability of the finding or from the imprecision of the replication.

References

A. Troy Adams; Howard Ajrouch, Kristine J.and Henderson & Heard Irene. Service-Learning Outcomes Research: The Role and Scarcity of Replication Studies. Journal of Applied Sociology, Volume 22, Number 2, Pages 55-74, 2005.
H. R. Barker & E. B. Gurman. Replication versus tests of equivalence. Perceptual and Motor Skills, Volume 35, Pages 807-814, 1972.
Cheryl Tatano Beck. Replication Strategies for Nursing Research. Journal of Nursing Scholarship, Volume 26, Number 3, Pages 191-194, Blackwell Publishing Ltd, 1994.
Kathleen B. Blomquist. Replication of research. Res. Nurs. Health, Volume 9, Number 3, Pages 193-194, Wiley Subscription Services, Inc., A Wiley Company, 1986.
S. W. Brown & K. A. Coney. Building a replication tradition in marketing. Marketing 1776-1976 and beyond, Chicago: American Marketing Association, 1976.
R.W. Easley; C.S. Madden & M.G. Dunn. Conducting Marketing Science: The Role of Replication in the Research Process. Journal of Business Research, April, Volume 48, Number 1, Pages 83-92, 2000.
Heiner Evanschitzky & J. Scott Armstrong. Replications of forecasting research. International Journal of Forecasting, Volume 26, Number 1, Pages 4 - 8, 2010.
B.M. Finifter. Replication and Extension of Social Research through Secondary Analysis. Social Science Information/Information sur les Sciences Sociales, Volume 14, Number 2, Pages 119-153, 1975.
B.M. Finifter. The Generation of Confidence: Evaluating Research Findings by Random Subsample Replication. Sociological Methodology, Volume 4, Pages 112-175, American Sociological Association, 1972.
Scott M. Fuess. On replications in business and economics research: the QJBE case. Quarterly Journal of Business and Economics, March, 1996.
Clyde Hendrick. Replications, Strict Replications, and Conceptual Replications: Are They Important?. , California, Pages 41-49, Sage, Newbury Park, 1990.
J.E. Hunter. The Desperate Need for Replications. Journal of Consumer Research, Volume 28, Number 1, Pages 149-158, 2001. H. H. Hyman & C. R. Wright. Evaluating social action programs. The Uses of Sociology, Pages 769-777, NY Basic, 1967.
M. H. van IJzendoorn. A process model of replication studies: on the relation between different types of replication. Leiden University Library, 1994.
Barry H. Kantowitz; Henry L. Roediger III & David G. Elmes. Experimental Psychology. , Pages 592, Wadsworth Publishing, 1984.
C.W. Kelly; L.J. Chase & R.K. Tucker. Replication in Experimental Communication Research: an Analysis. Human Communication Research, Volume 5, Number 4, Pages 338-342, 1979.
M. A. La Sorte. Replication as a Verification Technique in Survey Research: A Paradigm. The Sociological Quarterly, Volume 13, Number 2, Pages 218-227, Blackwell Publishing on behalf of the Midwest Sociological Society, 1972.
R.P. Leone & R.L. Schultz. A Study of Marketing Generalizations. The Journal of Marketing, Volume 44, Number 1, Pages 10-18, American Marketing Association, 1980.
R. M. Lindsay & A. S. C. Ehrenberg. The Design of Replicated Studies. The American Statistician, Volume 47, Number 3, Pages 217-228, American Statistical Association, 1993.
R. Lüdtke. Do that to me one more time! -- What kind of trial replications do we need?. Complementary Therapies in Medicine, Aug., Volume 16, Number 4, Pages 181-2, 2008.
D. T. Lykken. Statistical significance in psychological research.. Psychol Bull, Volume 70, Number 3, Pages 151-159, 1968.
R.A. Mittelstaedt & T.S. Zorn. Econometric Replication: Lessons from the Experimental Sciences. Quarterly Journal of Business & Economics, Volume 23, Number 1, 1984.
Kent B. Monroe. Front Matter. The Journal of Consumer Research, Volume 19, Number 1, Pages i-iv, The University of Chicago Press, 1992.
H. Radder. Experimental Reproducibility and the Experimenters' Regress. PSA: Proceedings of the Biennial Meeting of the Philosophy of Science Association, Volume 1, Pages 63-73, The University of Chicago Press on behalf of the Philosophy of Science Association, 1992.
S. Schmidt. Shall we really do it again? The powerful concept of replication is neglected in the social sciences.. Review of General Psychology, Stefan Schmidt, Academic Section Evaluation in Complementary and Alternative Medicine, Department of Environmental Health Sciences, University Medical Center Freiburg, Breisacherstr. 115b, Freiburg, Germany, D-79106, , Volume 13, Number 2, Pages 90-100, Educational Publishing Foundation: US, 2009.
M. Sidman. Tacticts of scientific research. , NY Basic, 1960. E.W.K. Tsang & K.-M. Kwan. Replication and Theory Development in Organizational Science: A Critical Realist Perspective. The Academy of Management Review, Volume 24, Number 4, Pages 759-780, Academy of Management, 1999.