“OBSERVATIONAL TECHNIQUES” Read the “Test Yourself” section on p. 147 in Ch. 6 of Exploring Research.
PART1- “OBSERVATIONAL TECHNIQUES” Read the “Test Yourself” section on p. 147 in Ch. 6 of Exploring Research.
Discuss your response with your classmates. However, an initial post is required.
Students need to contribute two substantive posts in this discussion by the due date indicated. The substantive posts can be any combination of responses and replies. However, an initial post is required.
Copyright 2018 by University of Phoenix. All rights reserved.
PART2- “ACHIEVEMENT TEST” Read the “Test Yourself” section on p. 142 in Ch. 6 of Exploring Research.
Discuss your response with your classmates.
PART3- “ONLINE LITERATURE REVIEW”
Read the “Test Yourself” section on p. 70 in Ch. 3A of Exploring Research.
Discuss your response with the class.
PART4- “HISTORY OF MODERN PSYCHOLOGY CHAPTER 6 & 7”
Read Chapters 6 & 7 in the History of Psychology Textbook and comment on at least 1 piece of history that interested you. Respond to one or more of the following prompts in one to two paragraphs:
Respond to one or more of the following prompts in one to two paragraphs:
1. Provide citation and reference to the chapter you discuss. Describe what you found interesting regarding this topic, and why.
2. Describe what you found interesting regarding this topic, and why.
3. Describe how you will apply that learning in your daily life, including your work life.
4. Describe what may be unclear to you, and what you would like to learn.
Running head: SETTING UP RESEARCH 1
Chapter 6 Methods of Measuring Behavior
WHAT YOU’LL LEARN ABOUT IN THIS CHAPTER:
· • The use of different methods of measuring behavior and collecting data
· • What a test is
· • How different types of tests are designed to assess different types of behavior
· • The use of achievement tests in the behavioral and social sciences
· • The design of multiple-choice items
· • How to do an item analysis
· • The application of attitude scales
· • The difference between Thurstone and Likert attitude scales
In Chapter 5 , you got a healthy dose of the theoretical issues that provide the foundation for the science of measurement, why measurement is crucial to the research process, how reliability and validity are defined, and how each of these can be established.
In this chapter, you will begin learning about the application of some of these principles as you read about different methods that can be used to measure behavior, including the ubiquitous test, the questionnaire, the interview, and other techniques.
As you read this chapter, keep several things in mind. Your foremost concern in deciding what method you will use to measure the behavior of interest should be whether the tool you intend to use is a reliable and valid one. This is equally true for the best-designed test and for the most informal-appearing interview. If your test does not “work,” then virtually nothing else will.
Second, the way in which you ask your question will determine the way in which you go about measuring the variables that interest you. If you want to know about how people feel toward a particular issue, then you are talking about attitudinal scales. If you want to know how much information people have about a particular subject, then you are talking about an achievement test or some other measure of knowledge. The focus of a study (such as the effects of unemployment on self-esteem) might be the same, whether you measure attitude or achievement, but what you use to assess your outcome variable depends on the question you ask. You need to decide the intent of your research activity, which in turn reflects your original research question and hypothesis.
Third, really efficient researchers are fully onboard for using whatever method helps them answer the questions that are being asked. This might include a mixed-methods model where one aspect of a research program might include qualitative methods while another might include qualitative methods (see Chapter 10 ). As research questions and their associated hypotheses become more intricate and complex, the creative side of using a particular research method correctly becomes more important.
Finally, keep in mind that methods vary widely in the time it takes to learn how to use them, in the measurement process itself, and in what you can do with the information once you have collected it. For example, an interview might be appropriate to determine how teachers feel about changes in the school administration, but interviewing would not be very useful if you were interested in assessing physical strength.
So, here is an overview of a variety of measurement tools. Like any other tool, use the one you choose well and you will be handsomely rewarded. Likewise, if you use the tool incorrectly, the job may not get done at all, and even if it does, the quality and value of your finished report will be less than what you expected.
The way in which you ask your research question will determine the method you use to assess the variables you are studying.
What better place to start than with the measurement method that all of us have been exposed to time and again: the good ol’ test?
Tests and Their Development
In the most general terms, the purpose of a test is to measure the nature and the extent of individual differences. For example, you might want to assess teenagers’ knowledge of how AIDS is transmitted. Or you may be interested in differences that exist on some measure of personality such as the Myers–Briggs Type Indicator or an intelligence test such as the Wechsler Intelligence Scales. Tests also are instruments that distinguish among people on such measures as reaction time, physical strength, agility, or the strategy someone selects to solve a problem. Not all tests use paper and pencil, and as we just mentioned, the technique that a researcher uses to assess a behavior often reflects that researcher’s creativity.
A good test should be able to differentiate people from one another reliably based on their true scores. Before continuing, here are just a few words of clarification. The word “test” is being used throughout this chapter to indicate a tool or technique to assess behavior but should not be used synonymously with the term “dependent variable.” Although you may use a test to assess some outcome, you may also use it for categorization or classification purposes. For example, if you want to investigate the effectiveness of two treatments (behavior therapy and medication, for example) on obsessive-compulsive disorders, you would first use the results of a test to categorize subjects into severe or mild categories and then use another assessment to evaluate the effectiveness of each treatment.
Why Use Tests?
Tests are highly popular in the assessment of social and behavioral outcomes because they serve a very specific purpose. They yield a score that reflects performance on some variable (such as intelligence, affection, emotional involvement, and activity level), and they can fill a variety of the researcher’s needs (summarized in Table 6.1 ).
First and foremost, tests help researchers determine the outcome of an experiment. Quite simply, tests are the measuring stick by which the effectiveness of a treatment is judged or the status of a variable such as height or voting preference in a sample is assessed. Because test results help us determine the value of an experiment, they can also be used to help us build and test hypotheses.
Second, tests can be used as diagnostic and screening tools, where they provide insight into an individual’s strengths and weaknesses. For example, the Denver Developmental Screening Test (DDST) assesses young children’s language, social, physical, and personal development. Although the DDST is a general screening test at best, it does provide important information about a child’s developmental status and areas that might need attention.
Third, tests assist in placement. For example, children who missed the date for kindergarten entrance in their school district could take a battery of tests to determine whether they have the skills and maturity to enter public school early. High school students often take advanced placement courses and then “test out” of basic required college courses. In these two cases, test scores assist when a recommendation is made as to where someone should be placed in a program.
Fourth, tests assist in selection. Who will get into graduate school is determined, at least in part, by an applicant’s score on tests such as the Graduate Record Examination (GRE) or the Miller’s Analogy Test (MAT). Businesses often conduct tests to screen individuals before they are hired to ensure that they have the basic skills necessary to complete training and perform competently.
Table 6.1 What tests do and how they do it
| What Tests Do | How Tests Do It | Examples |
| Help researchers determine the outcome of a study | Tests are used as dependent variables | A researcher wants to know which of two training programs is more effective |
| Provide diagnostic and screening information | Tests are usually administered at the beginning of a program to get some idea of the participant’s status | A teacher needs to know what type of reading program in which a particular child should be placed |
| Help in the placement process | Tests are used to place people in different settings based on specified characteristics | A mental health worker needs to place a client into a drug rehabilitation program |
| Assist in selection | Tests are used to distinguish between people who are admitted to certain programs | A graduate school committee uses test scores to make decisions about admitting undergraduates |
| Help evaluate outcomes | Tests are used to determine whether the goals of a program were met | A school superintendent uses a survey to measure whether the in-service programs had an impact on teachers’ attitudes |
Finally, tests are used to evaluate the outcomes of a program. Until you collect information that relates to the question you asked and then act on that information, you never really know whether the program you are assessing had, for example, the impact you sought. If you are interested in evaluating the effectiveness of a training program on returning war veterans, it is unlikely that you can judge the program’s efficacy without conducting some type of formal evaluation.
However, whether you use a test for selection or evaluation, it is not the test score that is in and of itself important, but rather the interpretation of that score. A score of 10 on an exam wherein all the items are simple is much different than a score of 10 where everyone else in the group received scores between 3 and 5.
Learning to design, create, administer, and score any test is important, but it is very important—and almost essential—to be able to know how to interpret that score.
What Tests Look Like
You may be most familiar with achievement-type tests, which often include multiple-choice items such as the following:
The cube root of 8 is
· a. 2
· b. 4
· c. 6
· d. 8
Multiple-choice questions are common items on many of the tests you will take throughout your college career. But tests can take on a variety of appearances, especially when you have to meet the needs of the people being tested and to sample the behavior you are interested in learning more about.
For example, you would not expect people with a severe visual impairment to take a pencil-and-paper test requiring them to darken small, closely placed circles. Similarly, if you want to know about children’s social interactions with their peers, you would probably be better served by observing them at play than by asking them about playing.
With such considerations in mind, you need to decide on the form a test might take. Some of the questions that will arise in deciding how a test should appear and be administered are as follows:
· • Is the test administered using paper and pencil, or is it administered some other way?
· • What is the nature of the behavior being assessed (cognitive, social, physical)?
· • Do people report their own behavior (self-report), or is their behavior observed?
· • Is the test timed, or is there no time limit?
· • Are the responses to the items subjective in nature (where the scoring is somewhat arbitrary) or objective (where there are clearly defined rules for scoring)?
· • Is the test given in a group or individually?
· • Are the test takers required to recognize the correct response (such as in a multiple-choice test) or to provide one (such as in a fill-in item or an open-ended question)?
TEST YOURSELF
Why test? Provide at least two reasons and an example of each.
Types of Tests
Tests are designed for a particular purpose: to assess an outcome whose value distinguishes different individuals from one another. Because many different types of outcome might be measured, there are different types of tests to do the job. For example, if you want to know how well a group of high school seniors understood a recent physics lesson, an achievement test would be appropriate.
On the other hand, if you are interested in better understanding the structure of an individual’s personality, a test such as the Minnesota Multiphasic Personality Inventory or the Thematic Apperception Test, two popular yet quite different tests of personality, would be more appropriate.
What follows is a discussion of some of the main types of tests you will run into in your research work, how they differ from one another, and how they can best be utilized.
Achievement Tests
Achievement tests are used to measure knowledge of a specific area. They are the most commonly used tests when learning is the outcome that is being measured. They are also used to measure the effectiveness of the instruction that accompanied the learning. For example, school districts sometimes use students’ scores on achievement tests to evaluate teacher effectiveness.
The spelling test you took every Friday in fourth grade, your final exam in freshman English, and your midterm exam in chemistry all were achievement tests administered for the same reason: they were designed to evaluate how well you understood specific information. Achievement tests come in all flavors, from the common multiple-choice test to true–false and essay examinations. All have their strengths and weaknesses.
Achievement tests are used to assess expertise in a content area.
There are basically two types of achievement tests: standardized tests and researcher-generated tests. Standardized tests , usually produced by commercial publishers, have broad application across a variety of different settings. What distinguishes a standardized test from others is that it comes with a standard set of instructions and scoring procedures.
For example, the Kansas Minimum Competency Test is a standardized test that has been administered to more than 2 million children across the state of Kansas in rural and urban settings, from very different social classes, school sizes, and backgrounds. Another example is the California Achievement Test (CAT), a nationally standardized test of achievement in the areas of reading, language, and arithmetic.
Researcher /Teacher-made tests, on the other hand, are designed for a much more specific purpose and are limited in their application to a much smaller number of people. For example, the test that you might take in this course would most likely be researcher or teacher made and designed specifically for the content of this course. Another example would be a test designed by a researcher to determine whether the use of teaching machines versus traditional teaching makes a difference in the learning of a foreign language.
Achievement tests can also be broken down into two other categories. Both standardized and researcher-made tests can be norm-referenced or criterion-referenced tests.
Norm-referenced tests allow you to compare an individual’s test performance to the test performance of other individuals. For example, if an 8-year-old student receives a score of 56 on a mathematics test, you can use the norms that are supplied with the test to determine that child’s placement relative to other 8-year-olds. Standardized tests are usually accompanied by norms, but this is usually not the case for teacher-made tests nor is the existence of norms a necessary condition for a test to be considered standardized. Remember, a test is standardized only if it has a standard or common set of administration and scoring procedures.
Criterion-referenced tests (a term coined by psychologist Robert Glaser in 1963) define a specific criterion or level of performance, and the only thing of importance is the individual’s performance, regardless of where that performance might stand in comparison with others. In this case, performance is defined as a function of mastery of some content domain. For example, if you were to specify a set of objectives for 12th-grade history and specify that students must show command of 90% of those objectives to pass, then you would be implying that the criterion is 90% mastery. Because this type of test actually focuses on the mastery of content at a specific level, it is also referred to as content-referenced testing.
When should you use which test? First, you must make this decision before you begin designing a test or searching for one to use in your research. The basic question you want to answer is whether you are interested in knowing how well an individual performs relative to others (for which norms are needed to make the comparison) or how well the individual has mastered a particular area of content (for which the mastery is reflected in the criterion you use).
Second, any achievement test, regardless of its content, can fall into one of the four cells shown in Table 6.2 , which illustrates the two dimensions just described: Does the test compare results with those of other individuals or to some criterion, and who designed or authored the test?
Multiple-Choice Achievement Items
Remember those endless hours filling in bubbles on optical-scanner scoring sheets or circling the A’s, B’s, C’s, and D’s, guessing which answer might be correct or not, and being told not to guess if you have no idea what the correct answer is? All these experiences are part of the multiple-choice question test, by far the most widely used type of question on achievement tests, and it is a type of test that deserves special attention.
Table 6.2 Classifying achievement tests as norm- or criterion-referenced and as standardized or researcher designed
The Anatomy of a Multiple-Choice Item
A multiple-choice question has its own special anatomy (see Figure 6.1 ). First, there is the stem , which has the purpose of setting the question or posing the problem. Second, there is the set of alternatives or options. One of these options must be the correct answer (alternative A in this example); the other three (in this example) should act as distracters .
The stem of a multiple-choice item should be written as clearly as possible to reduce method error.
A good distracter should be attractive enough that a person who does not know the right answer might find it plausible. Distracters that are far removed from reality (such as alternative d in Figure 6.1 ) are easily ruled out by the test taker and contribute to the lack of validity and reliability of the test. Why? Because the presence of poor distracters makes it even more difficult for the test to be an accurate estimator of a test taker’s true score.
What makes a great multiple-choice item? Any item that discriminates positively (where more people in the high group get it correct than people in the low group) is a potential keeper. Also we’d like that item to be relatively difficult, moving toward 50%. In sum, we want positively discriminating items with difficulty levels as close to 50% as possible.
To Use or Not to Use?
Multiple-choice questions are ideal for assessing the level of knowledge an individual has about a specific content domain, such as home economics, child development, geology, chemistry, Latin, fiber optics, sewing, or volleyball. But whatever the content of the test, the items must be written with the original objectives in mind of the lessons, chapters, papers, lectures, and other instruction to which the test takers were exposed.
Multiple-choice items have clear advantages and disadvantages.
If your Geology I professor did not have as an objective the distinction between different types of landforms, then items on distinguishing landforms should not be on the test. In other words, the content of a multiple-choice test should unequivocally reflect the content and objectives from which the items are drawn, and the number of items for each content area should reflect the amount of time spent on that content area during the teaching session.
Tests can take many different forms depending on their design and intended purpose.
Figure 6.1 The anatomy of a multiple-choice item.
In fact, many test creators use what is called a table of specifications which reflects the amount of teaching time as a function of topics so that if, for example, 20% of teaching time is spent on the basics of physical chemistry, the midterm should reflect that and have about 20% of the items on the basics.
There are several advantages and disadvantages to using multiple-choice items on an achievement test. These pros and cons should be taken into consideration if you intend to use such a test to assess a knowledge-based outcome. Here are some advantages of multiple-choice items:
· • They can be used to assess almost any content domain.
· • They are relatively easy to score and can be easily scored by machine.
· • Test takers do not have to write out elaborate answers but just select one of the test item’s alternatives.
· • Because multiple-choice items focus on knowledge and not on writing, people who are not good writers are not necessarily penalized for being unable to show what they know.
· • Good items are an investment in the future because they can be used over again, thus saving you preparation time.
· • Similarly, crummy items (you’ll find out what that is in a minute) can be discarded and no longer contribute to the unreliability of a test.
· • Good distracters can help a teacher diagnose the nature of the test taker’s failure to get the answer correct.
· • It is difficult to fake getting the answer correct, because the odds (such as .25 with four alternatives, including one correct answer) are stacked against it.
There are also some liabilities to multiple-choice items:
· • They may limit students’ options to generate creative answers.
· • There is no opportunity to evaluate writing skills.
· • Some people just do not like them and do not do well on them.
· • A multiple-choice type of question limits the kind of content that can be assessed.
· • Items must be very well written because bright students will detect poorly written alternatives and eliminate those as viable distracters.
Item Analysis: How to Tell If Your Items Work
A good multiple-choice item does one thing very well: It discriminates between those who know the information on the test and those who do not. For example, an item that everyone gets correct is of no use because it does not tell the examiner who knows the material and who does not. Similarly, an item that everyone gets wrong provides little information about the test takers’ understanding of the material. In other words, and in both cases, the item does not discriminate.
Wouldn’t it be nice if there were some numerical indices of how good a multiple-choice item really is? Wait no longer! Item analysis generates two such indices: difficulty level and discrimination level, which are two independent but complementary measures of an individual item’s effectiveness. Using these powerful tools, you can easily assess the value of an item and decide whether it should be kept in the item pool (the collection of multiple-choice items in a specific content area), revised, or tossed out!
Before either of these indices is computed, the total number of test scores has to be divided into a “high” group and a “low” group. To create these two groups, follow these steps:
· 1. Rank all the test scores from the highest to the lowest, so that the highest score is at the top of the list.
· 2. Define the high group as the top 27% of the test scores.
Item analysis results in a difficulty and discrimination index for each item on a test, not for the test as a whole.
· 3. Define the low group as the bottom 27% of the test scores. For example, if you have 150 adults in your sample, then the top 41 scores (or 27% of 150) would be in the high group, and the bottom 41 scores would be in the low group. Why is 27% the magic number? It is the amount that maximizes the discrimination between the two groups. If you recall, you want to compute the difficulty and discrimination indices to contrast groups of people who perform well with those who do not perform well.
· 4. For each item, examine the number of alternatives that were chosen by constructing the type of table you see in Table 6.3 . For example, 23 people in the high group selected alternative item A (which is the correct response) and 6 people in the low group selected alternative D.
The difficulty index is simply the proportion of test takers who got the item correct. The formula is
D = NC h + NC 1T
where D = difficulty level NC h = number of people in the high group who got the item correct NC 1 = number of people in the low group who got the item correct T = total number of people in the low and high groups
In this example, the difficulty level is
D = 23 + 1182 = .41
meaning that the difficulty level for that item is .41 or 41%, a moderately difficult item. (If everyone got the item wrong, the difficulty level would be 0%, and if everyone got the item correct, the difficulty level would be 100%.)
The discrimination index is a bit more complicated. It is the proportion of test takers in the upper group who got the item correct minus the proportion of test takers in the lower group who got the item correct. This value can range from +1.00 to −1.00. A discrimination index of −1.00 means that the item discriminates perfectly, and all the people in the high group got the item correct, whereas all the people in the low group got the item incorrect. Likewise, if the index is +1.00, this means that everyone in the low group got the item correct, whereas none of the high-scoring people got the item correct (not really the way it should be!).
To compute the discrimination index, use this formula:
d = NC h − NC 1( .5 ) T
Table 6.3 Data for computing the difficulty and discrimination indices of a multiple-choice item
where d = discrimination level NC h = number of people in the high group who got the item correct NC 1 = number of people in the low group who got the item correct T = total number of people in the low and high groups
In this example, the discrimination level is
d = 23 − 11( .5 ) 82 = .29
or 29%. You want items that discriminate between those who know and those who do not know but are not too easy or too hard.
Figure 6.2 shows the relationship between item discrimination and item difficulty. You can see that the only time an item can discriminate perfectly (1.00 or 100%) is when the item difficulty is 50%. Why? Because an item can discriminate perfectly only when two conditions are met. First, one-half of the group gets it right, and one-half of the group gets it wrong; second, the half that gets it right is in the upper half of those who took the test. As difficulty increases or decreases, discrimination is constrained.
You can work on the discrimination level as well as on the difficulty level in an effort to make your items better.
To change the difficulty level, try increasing or decreasing the attractiveness of the alternatives. If you change the attractiveness of the alternatives, you will find that the value of the discrimination will also change. For example, if an incorrect alternative becomes more attractive, it is likely that it will discriminate more effectively because it will fool those folks who almost—but not quite—know the right answer.
Computing these indices (by hand) can be a painstaking job, but using them is just about the only way you can tell whether an item is doing the job that it should. Many people who regularly use multiple-choice items suggest that you do the following to help track your items.
Each time you create an item, place it on a 3 inch × 5 inch index card. On the back of the card, enter the date of the test administration (and any other information you might deem important). Under the date, add any comments you might have and record the difficulty and discrimination indices for that particular test item. Then, as you work with these test items in the future, you will develop a file of test items with varying degrees of difficulty and discrimination levels. These items can be reused or altered as needed.
Figure 6.2 The relationship between item discrimination and item difficulty. Notice how item discrimination can be maximized only when item difficulty is 50%.
Figure 6.3 Data for computing the difficulty and discrimination indices of a multiple-choice item.
In order for you to discriminate between groups maximally, try to adjust the difficulty level of the item (which, to a large extent, is under the control of the researcher) so that it comes as close as possible to the 50% mark.
And instead of using index cards, create a spreadsheet for each test, where you can easily compute difficulty and discrimination indices for each item using simple formulas you create as shown in Figure 6.3 where the values of D and d are shown in the top of the example, and the actual formulas used to compute the values are shown in the bottom of the example.
If you chose to use a spreadsheet and formulas, just adjust the example in the table to fit your particular situation, such as the number of alternatives, the correct alternative, etc.
TEST YOURSELF
Achievement tests are ubiquitous in our society. Why do you think that’s the case?
Attitude Tests
Whereas achievement tests are probably the most commonly used type of test in our society (think of all those Friday afternoon spelling tests), other types are used in a variety of research applications. Among these are attitude tests , which assess an individual’s feelings about an object, person, or event. Attitude tests (sometimes called scales) are used when you are interested in knowing how someone feels about a particular thing, whether it be preference for a brand of microwave popcorn or feelings about euthanasia legislation.
For example, Figure 6.4 illustrates the basic format of a simple attitude scale. A statement is presented and then the individual indicates his or her attitude along some scale such as “Agree,” “No Strong Feeling,” and “Disagree.” The selection of items to be included and the design of the scale are tricky tasks that should not be undertaken lightly. Let’s look at two of the standard methodologies used for creating two types of scales, Thurstone and Likert, and see how they were developed.
Figure 6.4 A simple attitude scale.
Thurstone Scales
L. L. Thurstone was a famous psychometrician who developed the Thurstone scale , a method of measuring attitudes. He reasoned that if you could find out what value experts placed on a set of statements, then these statements could be scaled. People’s responses to these statements would indicate their attitude about the item in question. Here are the steps involved in the development of such a scale:
Thurstone scales come very close to measuring at the interval level.
· 1. As many statements as possible are written as potential test items. For example, if one were looking at parents’ attitudes toward their child’s school, some of these items might be
· a. I like the way my child’s teacher greets him or her in the morning.
· b. The principal does not communicate effectively with the teachers.
· c. My child’s education and potential are at risk.
· d. School lunches are healthy and nutritious.
· 2. Judges who are knowledgeable about the area of interest place the statements into 11 (actual physically different) stacks, ranging from the least favorable statement to the most favorable statement. Stack 6 (being right in the middle) represents a neutral statement. For example, item C below might be rated 1, 2, 3, 4, or 5 because it appears to be somewhat unfavorable.
· 3. Those statements rated consistently (with low variability) by judges are given the average score according to their placement. For example, if item A were rated as being 9 or 10 (somewhere around very favorable), it could receive a scale value of 9.5.
· 4. A group of statements then is selected that cover the entire range from unfavorable to favorable. That is your attitude scale.
One of the major advantages of Thurstone-like scales is that they are as close to the interval level of measurement (see Chapter 5 for a review of that idea) as one can get because the judges who rated the items placed them in stacks that have (presumably) equal distances between points which reflect psychological differences. It is for this reason that a Thurstone scale is also referred to as the method of equal-appearing intervals .
Respondents are asked to check off items on which they agree. Because the scale value assigned to the items that were checked off is known, an attitude score can be easily computed. If a person checks off many different items with scale values that are not approximately the same, then the individual’s attitude is not consistent or not well formed, or the scale has not been developed properly.
For example, here are some items on attitudes toward church from Thurstone and Chave’s classic work on attitude measurement, The Measurement of Attitudes (1929). Accompanying each item is its scale value.
· I believe the church is the greatest institution in America today (11)
· I believe in religion, but I seldom go to church (9.6)
· I believe in sincerity and goodness without any church ceremonies (6.7)
· I believe the church is a hindrance to religion for it still depends upon magic, superstition, and myth (5.4)
· I think the church is a parasite on society (.2)
It should be clear that the item with a scale value of 5.4 is more neutral in content relative (and that’s the really important term here) to any of the others.
Likert Scales
The Likert scale (Likert, 1932 ) is simple to develop and widely used. Although its construction is similar to a Thurstone scale, its development is less time consuming.
Likert scales are the most popular type of attitude assessment scale.
Here are the steps involved in the development of a Likert scale:
· 1. Statements are written that express an opinion or feeling about an event, object, or person. For example, if one were examining attitude toward federal support for child care, items might look like this:
· a. The federal government has no business supporting child care.
· b. Child care is an issue that the federal government should fully support.
· 2. Items that have clear positive and negative values (in the developer’s judgment) are selected.
· 3. The statements are listed, and to the right of each statement is a space for the respondent to indicate degree of agreement or disagreement, using a five-point scale such as:
SA Strongly agree
A Agree
U Undecided
D Disagree
SD Strongly disagree
Respondents are asked to circle or check their level of agreement with each item, as shown in Figure 6.5 .
Likert scales are scored by assigning a weight to each point along the scale, and an individual’s score is the average across all items. But it is not that simple. Because items can be reversed (such as where some are stated in the negative; for example, Government has no business funding child care programs), you must be consistent and reverse the scale when you score these items. The rule is that favorable items (such as Child care should be supported by federal, state, and local tax dollars) are rated 1–5, with 5 representing Strongly Agree. Unfavorable items are reversed in their scoring so that 1 represents Strongly Agree.
In the example in Figure 6.5 , the first item is written in the negative and the second one is written as a positive expression. Given the choices you see in Figure 6.5 , the scoring for these two items would be:
Figure 6.5 A set of Likert items.
producing a score of (2 + 1)/2 or 1.5, indicating a relatively strong level of general disagreement. Remember, item 1 was scored in reverse fashion because it is stated in the negative. And, remember that this is an abbreviated example using only two items. Because you sum ratings, the development of a Likert scale is often referred to as the method of summated ratings .
Personality Tests
Personality tests assess stable individual behavior patterns and are the most common type of test listed in the Buros Mental Measurement Yearbook (see http://www.unl.edu/buros ). Although personality tests can be very valuable assessment tools, they are extremely time consuming to develop and require training for the administration, scoring, and interpretation of the scores.
There are basically two types of personality tests: projective and structured tests. Projective tests present the respondent a somewhat ambiguous stimulus and then ask the person to formulate some type of response. The assumption underlying these types of tests is that the person being tested will project (or impose) his or her own view of the world on the stimuli and that these responses will form a pattern that can be evaluated by the trained person who is administering the test. These tests are unstructured.
Scoring these kinds of tests and reaching conclusions about personality patterns and behavior are not pie-in-the-sky stuff. Psychologists know that certain types of personalities respond in characteristic patterns; however, being able to recognize those patterns takes a great deal of time, training, and practice. Examples of these tests are the Thematic Apperception Test and the Rorschach Test.
Structured tests use a format that you might be familiar with, such as true–false, multiple choice, or yes–no. In these tests, people are asked to agree or disagree with an item that describes their feelings toward themselves (such as, “I like myself”). Examples of these tests are the Sixteen Personality Factor Questionnaire and the Minnesota Multiphasic Personality Inventory. One of the major advantages of the structured test over the projective test is that the structured test is objective in its item design and is easy to score. In fact, the publishers of these (and many other types of tests for that matter) offer scoring services. However, ease of scoring has nothing to do with interpreting the results of the test. Have no doubts, interpreting personality tests is best left to the experts who have the skills and the training. In fact, most publishers of personality tests will not sell you the materials until you can show proof of training (such as a Ph.D.) or have a trained person (such as your adviser) vouch for you.
There are all kinds of tests to test all kinds of things. What factors determine what kind of test you should use?
Observational Techniques
You may be most familiar with the type of test results that include an individual taking a test. That kind of test makes the respondent the active agent in the measurement process. In an entirely different class of behavior-assessment methods, the researcher (such as yourself) becomes the active agent. These are known as observational methods or observational techniques. In this technique, the researcher stands outside of the behavior being observed and creates a log, notes, or an audio or video record of the behavior.
For the most part, observers want to remove themselves from the action so that their presence does not affect the phenomena under observation.
Many terms are used to describe observational activity (several of which have been taken from the work done by anthropologists and ethnologists), such as fieldwork or naturalistic observation. The most important point to remember about observational methods is why they have been so useful to scientists in other disciplines; their primary goal is to record behavior without interference. As an observer, you should make every effort to stay clear of the behavior you are observing so that you are unobtrusive and do not interfere.
For example, if you are interested in studying play behavior among children with disabilities and those without disabilities, you will be well served to observe these children from afar rather than to become a part of their setting. Your presence while they play would undoubtedly have an impact upon their behavior.
You can find a great deal of additional information on observational techniques in Chapter 10 , which covers qualitative methods and discusses such techniques as ethnographic research and case studies.
Techniques for Recording Behavior
Several different techniques can be used to observe and record behavior in the field. They fall into four general categories: duration recording, frequency recording, interval recording, and continuous recording.
In the first category, duration recording , the researcher uses a device to keep track of time and measures the length of time that a behavior occurs. For example, the researcher might be interested in knowing how much physical activity occurs during kindergarteners’ morning recess. The researcher might use a stopwatch to record the length of time that physical activity takes place for one child, then go on to another child, and so forth. The researcher is recording the duration of a particular event.
The second major technique category for observing behavior is frequency recording , in which the incidence or frequency of the occurrence of a particular behavior is charted. For example, a researcher might want to record the number of times that a shopper picks up and feels the fabric of which clothes are made or the number of comments made about a particular brand of soap.
A third category is interval recording or time sampling , in which a particular subject is observed during a particular interval of time. For example, the researcher might observe each child in a play group for 15 seconds, record the target behaviors, and then move on to the next child for 15 seconds. Here, the interval deals with the time the observer focuses on a particular subject, regardless of what the subject might be doing.
Finally, in continuous recording , all of the behavior of the target subject is recorded, with little concern as to the specificity of its content. Often, people who complete case studies observe a child for a particular length of time and have no previously designated set of behaviors for which to look. Rather, the behaviors that are recorded are those that occur in the natural stream of events. This is a rich and fruitful way of collecting information, but it has a major disadvantage: The little planning that goes into recording the information necessitates intensive sifting through of the records at analysis time.
Table 6.4 provides a summary and gives an example of these four different kinds of techniques—and what each kind does.
There are a few potentially unattractive things about the use of these techniques. Primarily (you’ve just read this but it’s important enough to repeat), the very act of observing some behaviors interferes with the actual behavior that researchers may want to study. For example, have you ever walked into an elementary school classroom and noticed that all the children look at you? Some children may even put on a bit of a show for you. Sooner or later that type of behavior on the part of the children would settle down, but you certainly are not going to get an uninfluenced view of what occurs there.
The key word, then, is “unobtrusive”—observing behavior without changing the nature of what is being observed.
The use of these four different techniques has been eased greatly by the introduction and availability of easy-to-use technology. For example, you need not sit and continuously observe a group of adults making a decision when you can videotape the group and then go back to do an in-depth analysis of their behaviors. Similarly, rather than using a pencil and paper to record behavior every 10 seconds, you can use your iPhone or other smartphone (appropriately programmed) to beep every 10 seconds and then press a key to enter the category of the behavior.
Remember that any such collection of data needs to be done with particular attention given to such concerns as anonymity and respect for the person being observed (addressed in Chapter 3B ). For example, you have to pick and choose where and how you do your observing. Although it might be very interesting to listen in on the private talk of adolescents in the restroom, it also might be a violation of their right to privacy. Recording phone conversations might be an effective way to assure anonymity, because you might not know the caller’s name (if you solicit callers), but people need to be notified when conversations are being recorded (remember Watergate, Linda Tripp, and other famous folks).
Table 6.4 Four ways to observe and record behaviour
| Technique | How It Works | Example |
| Duration recording | The researcher records the length of time that a behavior occurs | How much time is spent in verbal interaction between two children? |
| Frequency recording | The researcher records the number of times a behavior occurs | How often are questions asked? |
| Interval recording | The researcher observes a subject for a fixed amount of time | Within a 60-second period, how many times do members of the group talk to another person? |
| Continuous recording | The researcher records everything that happens | During a 1-hour period, all the behavior of a 6-year-old boy is recorded |
Observational Techniques? Be Careful!
No technique for assessing behavior is perfect, and all are fraught with potential problems that can sink your best efforts. Some particular problems that you should consider if you want to use observational techniques are as follows:
· • Your very presence may affect the behavior being observed.
· • Your own bias or viewpoints might affect the way in which you record behavior, from what you select to record to the way you do the recording.
· • You may become fatigued or bored and miss important aspects of the behavior being recorded or miss the behavior itself.
· • You may change the definition of those behaviors you want to observe such that what was defined as aggression at time 1 (touching without permission) is redefined at time 2 because you realize that all touching (even without permission) is not necessarily aggressive.
There are a few good reasons why one should be very careful when using observational techniques mostly having to do with contamination by the observer.
TEST YOURSELF
Provide an example of where the person doing the examining might interfere with the measurement itself.
Questionnaires
Questionnaires are (most often) a paper-and-pencil set of structured and focused questions. Questionnaires save time because individuals can complete them without any direct assistance or intervention from the researcher (many are self-administered).
In fact, when you cannot be with participants personally, a mailed questionnaire can produce the data you need.
There are other advantages to questionnaires besides their being self-administered:
· • By using snail mail or e-mail, you can survey a broad geographical area. Also relatively new to the world of doing survey research are online, Web-based survey tools such as SurveyMonkey (at surveymonkey.com ), Zoomerang (at zoomerang.com ), and SurveyGizmo (at surveygizmo.com ). These are all free in the limited version. For example, SurveyMonkey allows for 100 responses and no customization or downloading, but for $20 per month, there are no limits. For a large-scale research project, where data is only collected for a few months, this can be a huge help and savings of time and effort.
· • They are cheaper (even with increased postage costs) than one-on-one interviews.
· • People may be more willing to be truthful because their anonymity is virtually guaranteed.
The objectivity of the data also makes it easy to share with other researchers and to use for additional analysis. Although the time that the data were collected may have passed, answers to new questions beyond those originally posed might just be waiting to be answered.
For example, in one study, S. L. Hanson and A. L. Ginsburg ( 1988 ) used the results of the High School and Beyond surveys originally collected in the spring of 1980 from more than 30,000 sophomore students. These researchers were interested in examining the relationships among high school students’ values, test scores, grades, discipline problems, and dropout status. With an original 84% response rate, these surveys provide a large, comprehensive database. The response rate may have been unusually high because the students were probably part of a captive audience. In other words, they were given the questionnaires as part of regular school activities.
Keep in mind, however, that all these advantages are not necessarily a recommendation to go out and start collecting all your data using this method. One of the big disadvantages of questionnaires is that the completion and return rates are much lower than if you personally asked the questions of each potential respondent through an interview, a technique you will get to shortly. Although you would expect a high participation rate (up to 100%) if you were to visit people’s homes and ask them questions, you can expect about a 35% return rate on a mailed questionnaire.
What Makes Questionnaires Work
What’s a good questionnaire? Several factors make a questionnaire successful, or result in a high number of returns with all the items (or as many as possible) completed. You have completed questionnaires at one time or another, whether they were about your attitude toward the 2004 Green Grass Party ticket or what you want in a stereo receiver. Whether or not the questionnaires work depends on a variety of factors under your control. Let’s look at a brief discussion of each of these factors, which are summarized in Table 6.5 and broken down into three general parts: the basic assumptions on which the questionnaire is based, the questions themselves, and the format in which the items are presented.
Questionnaires are very useful, but they take a lot of time and effort to develop.
Basic Assumptions of the Questionnaire
There are five important points regarding the basic assumptions that one makes when designing a questionnaire. Possible respondents are probably quite willing to help you, but you must help them to be the kind of respondent you want.
· 1. You would not ask respondents to complete a 40-page questionnaire or to take 3 hours on Saturday to do it. Your questionnaire must be designed in such a way that its demands of time, expense, and effort are reasonable. You also want to avoid asking questions that are inappropriate (too personal) or phrased in the wrong way. Anything that you would find offensive will probably offend your potential respondents as well.
Table 6.5Some important things to remember about the design and use of questionnaires
| The Basic Assumptions· • The questionnaire does not make unreasonable demands upon the respondent· • The questionnaire does not have a hidden purpose· • The questionnaire requests information that respondents presumably have |
| The Questions· • The questionnaire contains questions that can be answered· • The questionnaire contains questions that are straightforward |
| The Format· • The items and the questionnaire are presented in an attractive, professional, and easy-to-understand format· • All questions and pages are clearly numbered· • The questionnaire contains clear and explicit directions as to how it should be completed and how it should be returned· • The questions are objective· • The questions are ordered from easy to difficult and from easy to specific· • Transitions are used from one topic to the next· • Examples are given when necessary |
· 2. Your questionnaire must be designed to accomplish your goal, not to collect information on a related but implicit topic. If you are interested in racial attitudes, then you should direct your questions to racial attitudes and not ask questions framed within a different context that is related, but not central, to your purpose.
· 3. If you want to find out about a respondent’s knowledge of some area, you must assume that the person has the knowledge to share. Asking a first-semester freshman on the first day of classes about the benefits of college would probably not provide meaningful data. However, on a student’s last day of college, you would probably get a gold mine of information.
· 4. Encourage respondents by designing a questionnaire that contains interesting questions, that engages respondents in answering all your questions, and that prompts them to return the questionnaire to you. If you cannot make your questions interesting, perhaps you do not have enough knowledge or enthusiasm about the topic and you should select another topic.
· 5. If you can get the same information through a source other than a questionnaire, by all means do so. If an interview gets you a better response and more accurate data, use an interview. If you can find out someone’s GPA through another source, it’s better to take the extra time necessary than to load the respondent with issues that really are secondary to your purpose.
What About the Questions?
Questions come in all shapes and sizes, and some are absolutely terrible. For example: Do you often feel anxious about taking a test and getting a low grade?
Can you see why this is not a good question? To begin with, the and makes it two questions rather than one, making it very difficult to know what the respondent was reacting to. Designing good questions takes some time and practice.
If you are using a questionnaire, be reasonable when planning the what, when, where, and how of your research plan.
First, be sure the questions you ask can be answered. Do not ask about a person’s attitude toward political strife in some foreign country if they know nothing about the country’s state of affairs.
Similarly, ask the question in a straightforward manner: for example, Do you never not cheat on your exams? This question is convoluted, uses a double negative, and is just as easily asked as, Do you ever cheat on your exams? This form is clearer and easier to answer accurately.
Finally, take into account the social desirability of questions. Will anyone graciously and positively answer the question, Do you beat your children? Of course not, and information from such direct questions may be of questionable value.
The Format of the Questionnaire
As you can see in Table 6.5 , several criteria can be applied to the format of a questionnaire, and each one of them is so important that glossing over it could sink your entire project.
For example, let’s say that you create this terrific questionnaire with well-designed questions, and you allow just the right amount of time for completion, and you even call all the participants to see if they have any questions. Unfortunately, you forget to give them detailed instructions on how to return it to you! Or perhaps you include clear return instructions but forget to tell them how to answer the questions.
Don’t underestimate the appearance of the instruments you use. Neat and tidy helps increase reliability.
· • If your questionnaire does not consist of items or questions that are easy to read (clearly printed, not physically bunched together, etc.), you will get nowhere fast. The items must be neatly arranged, and the entire questionnaire must be clearly duplicated. Almost any word processing program contains templates that can help you with such considerations as white space, proportion, and balance, so you end up with a professional-looking document.
· • All questions and pages should be plainly numbered (e.g., 1, 2, 3, 4 . . . ). Do not use cumbersome or potentially confusing combinations such as I-1.2 or II.4.
· • Good questionnaires contain directions that are complete and to the point. They tell the respondent exactly what to do (“complete this section”) and how to do it (“circle one answer,” “check all that apply”). These directions also offer explicit directions as to how the questionnaire should be returned, including pread-dressed stamped envelopes and a phone number to call for more information if necessary.
· • Your respondents are doing you a favor by completing the questionnaire. Your goal is to get as many as possible to do just that. One way to encourage responses is to show that your work is supported by a faculty member or your adviser, which you can do through a cover letter like the one you saw in Figure 3B.1 .
· • You want as honest an answer as possible from your respondents and, consequently, you must be careful that your questions are not leading them to answer in a particular direction. Questions must be objective and forthright. Once again, be careful of socially undesirable statements.
· • Initial questions should warm up the respondent. In the beginning, relatively simple, nonthreatening, and easy-to-answer questions (“How old were you on your last birthday?”) should be presented to help the respondent feel comfortable. Then as the questions progress, more complicated (and personal) questions might be asked. For example, many questionnaires begin with questions about demographics such as age, gender, race, and so on, all information that most people find relatively nonintimidating to provide. Subsequent questions might deal with issues such as feelings toward prejudice, questions about religion, and the like.
· • When your questionnaire changes gears (or topics), you have to let the respondent know. If there is a group of questions about demographics followed by a set of questions about race relations, you need a transition from one to the other; for example: “Thank you for answering these questions about yourself. Now we would like to ask you some questions about your experiences with people who are from the same ethnic group as you as well as from other groups.”
· • Finally, make every effort to design a questionnaire that is easy to score. When possible, provide answer options that are objective and close ended, such as 27. What is your annual income?
· a. Below $20,000
· b. $20,000 to $24,999
· c. $25,000 to $29,999
rather than
27. Please enter your annual income: $———.
In the first example, you can enter a code representing the letter as the response to be used for later analysis. In the second, you must first record the number entered and then place it in some category, adding an extra step.
The Cover Letter
An essential part of any questionnaire is the accompanying cover letter. This message is important because it helps set the scene for what is to come. A good cover letter is especially important for questionnaires that are mailed (snail mail or e-mail) to respondents so that the sense of authority is established and the importance of the project is conveyed. Here are some tips on what a good cover letter should contain:
A good cover letter can make or break the success of a project.
· • It is written on official letterhead, which helps favorably to impress respondents and increases the likelihood that they will respond.
· • It is dated recently, thus indicating that there is some urgency to the request.
· • It is personalized; it opens by stating “Dear Mr. and Mrs. Margolis,” not “Dear Participant.”
· • It clearly states the purpose of the questionnaire and the importance of the study.
· • It gives a time estimate so respondents know when to return it.
· • It clearly promises confidentiality and indicates how confidentiality will be ensured.
· • It makes respondents feel that they are part of the project in that a copy of the results will be sent to them when the study is complete.
· • It includes a clear, physically separate expression of thanks.
· • It is signed by the “big boss” and by you. Although you would like to stand on your own name and work, at this early point in your career this little bit of help from the boss can make an important difference.
Summary
In our society, tests for everything from selection to screening are everywhere, and their use has become one of the most controversial topics facing social and behavioral scientists. Tests definitely have their place, and in this chapter different kinds of measurement tools and how they can be used to reliably and validly assess behavior has been discussed. Remember, however, that careful formulation of hypotheses and attention to detail throughout the research project are also required for your measurement method to yield an accurate result.
Exercises
1 .
Before beginning the test, individuals taking one of the Wechsler Intelligence Scales are informed that some questions will be easy, some questions will be difficult, and test takers are not expected to be able to answer every question. Why is this information a sign of a good test?
2 .
What is a standardized test? How does it differ from a researcher- or teacher-made test?
3 .
A psychology licensing examination requires individuals to answer 70% of questions correctly in order to receive a passing score. (a) Is this exam norm-referenced or criterion-referenced? (b) What if the test required individuals to perform better than 70% of individuals who took the test in order to receive a passing score?
4 .
How can you change the difficulty level of your multiple-choice test items?
5 .
For the following set of information about two achievement test scores, compute the difficulty and the discrimination indices. The asterisk corresponds to the correct answer.
| Item 1 | ||||
| Alternatives | A* | B | C | D |
| Upper 27% | 28 | 6 | 7 | 20 |
| Lower 27% | 6 | 12 | 21 | 22 |
| Item 2 | ||||
| Alternatives | A* | B | C | D |
| Upper 27% | 10 | 7 | 28 | 15 |
| Lower 27% | 15 | 0 | 15 | 30 |
6 .
Write a 10-item questionnaire (using Likert-type items) that measures attitude toward stealing. Be sure to use both positive and negative statements and state all of the items simply enough so that they can be easily answered. Also, be sure to include a set of instructions. Give your work a day or two rest and then have a classmate check it over.
7 .
Consult the latest edition of Buros Mental Measurements Yearbook (online or in the library) and summarize a review of any test that is mentioned. What was the purpose of the test? Is the review positive or negative? How can the test be improved?
8 .
Interpret the following discrimination and difficulty scores:
· (a)D = .50
d = − .90
· (b)D = .90
d = .25
9 .
Describe three basic characteristics of a questionnaire.
10 .
What are three advantages of using a questionnaire?
11 .
Consider the following response set on a Likert scale with 1 representing Strongly Disagree and 5 representing Strongly Agree. What total score would you give this respondent (responses are in bold)?
· 1.The actors in the movie performed well. 1 2 3 4 5
· 2.The plot was interesting. 1 2 3 4 5
· 3.Most of the scenes in the movie were boring. 1 2 3 4 5
· 4.Overall, the movie was enjoyable. 1 2 3 4 5
Online. . .
Educational Testing Service
The Educational Testing Service ( http://www.ets.org/ ) is the home page for the mother of all commercial test services. These are the people who bring you the SATs, GREs, APs, and more. They score the tests, send the results where you want, and even help you understand why you didn’t get that perfect 800 on the math portion. This is a good site for general information about testing, financial aid, and other college-related topics.
FairTest: The National Center for Fair & Open Testing
FairTest ( http://www.fairtest.org/ ) is an advocacy organization that works to end the misuses and flaws they claim are inherent in standardized testing and make sure that the testing process is fair. This entire area of tests and their fairness is very controversial and well worth learning about.
Rethinking Schools
Why the testing craze won’t fix our schools. See Rethinking Schools Online at http://www.rethinkingschools.org/ , for a collection of articles on testing and assessment, from the perspective of the classroom teacher and students.
Personality Tests
Play around on this site ( http://www.similarminds.com/personality_tests.html ) by SimilarMinds to learn about different personality types and personality research. The site also has personality tests available for taking, but keep in mind that personality tests are best administered by a professional with the appropriate credentials.
Chapter 3A Selecting a Problem and Reviewing the Research
WHAT YOU’LL LEARN ABOUT IN THIS CHAPTER:
· • How to select a research problem
· • Defining and sorting out idea after idea until one fits your interests
· • The importance of personal experience in selecting a problem
· • The steps in reviewing the literature
· • Different sources of information and how to use them
· • How to use journals, abstracts, and indices
· • The difference between primary and secondary resources
· • Using a synthesis of literature
· • How scholarly journals work
· • Using the Internet to complete your literature review
So here you are, in the early part of a course that focuses on research methods, and now you have to come up with a problem that you are supposed to be interested in! You are probably so anxious about learning the material contained in your professor’s lectures and what is in this volume that you barely have time to think about anything else.
If you stop for a moment and let your mind explore some of the issues in the behavioral and social sciences that have piqued your interest, you will surely find something that you want to know more about. That is what the research process is all about—finding out more about something that is, in part, already known.
Once you select an area of interest, you are only part of the way there. Next comes the statement of this interest in the form of a research question followed by a formal hypothesis. Then it is on to reviewing the literature, a sort of fancy phrase that sounds like you will be very busy! A literature review involves library time online or actually there, note taking, and organizational skills (and of course writing), but it provides a perspective on your question that you cannot get without knowing what other work has been done as well as what new work needs to be done.
But hold on a minute! How is someone supposed to have a broad enough understanding of the field and spew forth well-formed hypotheses before the literature is reviewed and then become familiar with what is out there? As poet John Ciardi wrote, therein “lies the rub.”
The traditional philosophers and historians of science would have us believe that the sequence of events leading up to a review of what has been done before (as revealed in the literature) is as shown in Figure 3A.1a . This sequence of steps is fine in theory, but as you will discover, the actual process does not go exactly in the manner shown in the figure.
The research question and research hypothesis are more an outgrowth of an interaction between the scientist’s original idea and an ongoing, thorough review of the literature (good scientists are always reading), as you can see in Figure 3A.1b . This means that once you formulate a hypothesis, it is not carved in stone but can be altered to fit what the review of the literature may reflect, as well as any change in ideas you may have. Remember, our work “stands on the shoulder of grants.”
For example, you might be interested in how working adults manage their time when they are enrolled in graduate programs. That’s the kernel of the idea you want to investigate. A research question might ask what the effects of enrollment in graduate school and full-time work are on personal relationships and personal growth. For a hypothesis, you might predict that those adults enrolled in school and who work full time and who participate in a time management support group have more meaningful personal relationships than those who do not.
Figure 3A.1a From idea to literature review, with the research hypothesis on the way.
Figure 3A.1b From idea and literature review to research hypothesis.
Use the results of previous studies to fine-tune your research ideas and hypothesis.
You might consider the hypothesis to be finished at this point, but in reality your ongoing review of the literature and your changing ideas about the relationship between the variables will influence the direction your research will take. For example, suppose the findings of a similar previous study prompt you to add an interesting dimension (such as whether the employer subsidizes the cost of tuition) to your study, because the addition is consistent with the intent of your study. You should not have to restrict your creative thinking or your efforts to help you understand the effects of these factors just because you have already formulated a hypothesis and completed a literature review. Indeed, the reason for completing the review is to see what new directions your work might take. The literature review and the idea play off one another to help you form a relevant, conceptually sound research question and research hypothesis.
In sum, you will almost always find that your first shot at a hypothesis might need revision, given the content of the literature that you review. Remember, it is your idea that you will pursue. The way in which you execute it as a research study will be determined by the way in which you state the research question and the way in which you test the research hypothesis. It is doubtful that a review of the relevant literature would not shed some light on this matter.
This chapter begins with some pointers on selecting a problem worth studying, and then the focus moves to a description of the tools and the steps involved in preparing a review of the literature.
Selecting a Problem
People go to undergraduate and graduate school for a variety of reasons, including preparing for a career, the potential financial advantages of higher education, and even expanding their personal horizons and experiencing the sheer joy of learning (what a radical thought!). Many of you are in this specific course for one or more of these reasons.
Select a problem which genuinely interests you.
The great commonality between your course work and activities is your exposure to a wealth of information which you would not otherwise experience. That is the primary purpose of taking the time to select a research problem that makes sense to you and that interests you, while at the same time makes a contribution to your specific discipline. The selection of the area in which to work on is extremely important for two reasons. First, research takes a great deal of time and energy, and you want to be sure that the area you select interests you. You will work so hard throughout this project that continuing to work on it, even if it’s the most interesting project, may at times become overwhelming. Just think of what it would be like if you were not interested in the topic!
Second, the area you select is only the first step in the research process. If this goes well, the remaining steps, which are neither more nor less important, also have a good chance of going well.
Just as there are many different ways to go about selecting a research problem, there are also some potential hazards. To start you off on the right foot, the following briefly reviews some of these almost fatal errors.
It is easy to do, but falling in love with your idea can be fatal. This happens when you become so infatuated with an idea and the project and you invest so much energy in it that you cannot bear to change anything about it. Right away someone is going to say, “What’s wrong with being enthusiastic about your project?” My response is a strong, “Nothing at all.” As does your professor, most researchers encourage and look for enthusiasm in students (and scientists) as an important and essential quality. But enthusiasm is not incompatible with being objective and dispassionate about the actual research process (not the content). Sometimes—and this is especially true for beginning research students—researchers see their question as one of such magnitude and importance that they fail to listen to those around them, including their adviser, who is trying to help them formulate their problem in such a way as to make it more precise and, in the long run, easier to address. Be committed to your ideas and enthusiastic about your topic but not so much that it clouds your judgment as to the practical and correct way to do things.
Next, sticking with the first idea that comes to mind isn’t always wise. Every time the 1930s cartoon character Betty Boop had a problem, her inventor grandfather would sit on his stool, cross his legs (taking a Rodin-like pose), and think about a solution. Like a bolt from the blue, the light bulb above his head would go on, and Grampy would exclaim, “I’ve got it!,” but the idea was never exactly right. Another flash would occur, but once again the idea was not perfect. Invariably, it was the third time the light went on that he struck gold.
Do you like your first idea for a research study? Great, but don’t run out and place an advertisement for research participants in the newspaper quite yet. Wait a few days and think about it, and by no means should you stop talking to other students and your adviser during this thinking stage. Second and third ideas are usually much more refined, easier to research, and more manageable than first ones. As you work, rewrite and rethink your work . . . constantly.
Do you want to guarantee an unsuccessful project that excites no one (except perhaps yourself)? Doing something trivial by selecting a problem that has no conceptual basis or apparent importance in the field can lead to a frustrating experience and one that provides no closure. Beginning students who make this mistake sometimes over-intellectualize the importance of their research plans and don’t take the time to ask themselves, “Where does this study fit in with all that has been done before?” Any scientific endeavor has as its highest goal the contribution of information that will help us better to understand the world in general and the specific topic being studied in particular. If you find out what has been done by reading previous studies and use that information as a foundation, then you will surely come up with a research problem of significance and value.
Ah, then there are researchers who bite off more than they can chew. Sound silly? Not to the thousands of advisers who sit day after day in their offices trying to convince well-intentioned beginning students that their ideas are interesting but that (for example) it may be a bit ambitious to ask every third adult in New York City about their attitudes toward increasing taxes to pay for education. Grand schemes are fine, but unless you can reduce a question to a manageable size, you might as well forget about starting your research. If these giant studies by first-timers ever do get done (most of the time they don’t in their original form), the experiences are usually more negative than positive. Sometimes these students end up as ABD s (all but dissertation). Although you may not be seeking a doctorate right now, the lesson is still a good one. Give yourself a break from the beginning—choose a research question that is doable.
Finally, if you do something that has already been done, you could be wasting your time. There is a fine line between what has been done and what is important to do next based on previous work. Part of your job is to learn how to build and elaborate on the results of previous research without duplicating previous efforts. You might remember from the beginning of this chapter that I stressed how replication is an important component of the scientific process and good research. Your adviser can clearly guide you as to what is redundant (doing the exact same thing over without any sound rationale) and what is an important contribution (doing the same thing over but exploring an aspect of the previous research or even asking the same question, while eliminating possibly confounding sources of variance present in the first study).
TEST YOURSELF
Perhaps one of the most interesting dimensions of being a scientist is how the questions they ask are modified as they review the literature and learn more about the topic they are interested in. It’s a constant give and take—hence the importance of being well informed. Ask your advisor or some other faculty how they keep themselves informed in their own field of study.
Defining Your Interests
It is always easy for accomplished researchers to come up with additional ideas for research, but that is what they are paid and trained to do (in part, anyway). Besides, experienced researchers can put all that experience to work for themselves, and one thing (a study) usually leads to another (another study).
Never disregard personal experience as an important source of ideas.
But what about the beginning student such as yourself? Where do you get your ideas for research? Even if you have a burning desire to be an experimental psychologist, a teacher, a counselor, or a clinical social worker, where do you begin to find hints about ideas that you might want to pursue?
In some relatively rare cases, students know from the beginning what they want to select as a research area and what research questions they want to ask. Most students, however, experience more anxiety and doubt than confidence. Before you begin the all-important literature review, first take a look at the following suggestions for where you might find interesting questions that are well worth considering as research topics.
First, personal experiences and firsthand knowledge more often than not can be the catalyst for starting research. For example, perhaps you worked at a summer camp with disabled children and are interested in knowing more about the most effective way to teach these children. Or, through your own personal reading, you have become curious about the aging process and how it affects the learning process. A further example: At least three of my colleagues are special educators because they have siblings who were not offered the special services they needed as children to reach their potential. Your own experiences shape the type of person you are. It would be a shame to ignore your past when considering the general area and content of a research question, even if you cannot see an immediate link between these experiences and possible research activities. Keep reading and you will find ways to help you create that link.
You may want to take complete responsibility for coming up with a research question. On the other hand, there is absolutely nothing wrong with consulting your advisor or some other faculty member who is working on some interesting topic and asking, “What’s next?” Using ideas from your mentor or instructorwill probably make you very current with whatever is happening in your field. Doing so also will help to establish and nurture the important relationship between you and your adviser (or some other faculty member), which is necessary for an enjoyable and successful experience. These are the people doing the research, and it would be surprising not to find that they have more ideas than time to devote to them and would welcome a bright, energetic student (like you) who wants to help extend their research activities.
Next, you might look for a research question that reflects the next step in the research process. Perhaps A, B, and C have already been done, and D is next in line. For example, your special interest might be understanding the lifestyle factors that contribute to heart disease, and you already know that factors such as personality type (for example, Type A and Type B) and health habits (for example, social drinking) have been well studied and their effects well documented. The next logical step might be to look at factors such as work habits (including occupation and attitude) or some component of family life (such as quality of relationships). As with research activities in almost all disciplines and within almost all topics, there is always that next logical step that needs to be taken.
Last, but never least, is that you may have to come up with a research question because of this class. Now that is not all that bad either, if you look at it this way: People who come up with ideas on their own are all set and need not worry about coming up with an idea by the deadline. Those people who have trouble formulating ideas need a deadline; otherwise, they would not get anything done. So although there are loftier reasons for coming up with research questions, sometimes it is just required. Even so, you need to work very hard at selecting a topic that you can formulate as a research question so that your interest is held throughout the duration of the activity.
TEST YOURSELF
You’d be surprised how many important scientific breakthroughs were the result of informal talk (aka “bull”) sessions between people interested in the same or similar topics. Just sitting around and talking about ideas is one of the great pleasures when it comes to learning and scientific discovery. Be a bit creative and list five ideas you have or questions you find particularly interesting about any topic. Don’t worry at this point how you would answer the question but take a few intellectual risks and see what you come up with.
Ideas, Ideas, Ideas (and What to Do with Them)
Even if you are sure of what your interest might be, sometimes it is still difficult to come up with a specific idea for a research project. For better or worse, you are really the only one who can do this for yourself, but the following is a list of possible research topics, one of which might strike a chord. For each of these topics, there is a wealth of associated literature. If one topic piques your interest, go to that body of literature (described in the second part of this chapter) and start reading.
· aggression
AIDS
autism spectrum disorder
bilingual education
biofeedback
biology of memory
birth control
body image
central nervous system
child care
children of war
circadian rhythms
classical conditioning
cognitive development
color vision
competition
compliance
computer applications
conflict
cooperative learning
creativity
delusions
depression
déjà vu
development of drawing
diets
divorce
dreams
drug abuse
early intervention
egocentrism
elder care
endocrine system
epilepsy
ethics
exercise
fat
fetal alcohol syndrome
fluid intelligence
gender differences
Head Start
homeschooling
identity
imagery
intelligence
language development
learning disabilities
mediation
memory
menarche
mental sets
middle adulthood
motivation
narcolepsy
neural development
nightmares
nutrition
obesity
optimism
pain
parenting
perception
prejudice
public policy
racial integration
reinforcement
relaxation
REM sleep
self-esteem
violence in schools
From Idea to Research Question to Hypothesis
Once you have determined what your specific interest might be, you should move as quickly as possible to formulate a research question that you want to investigate and begin your review of literature.
Research ideas lead the way to hypotheses.
There is a significant difference between your expressing an interest in a particular idea and the statement of a research question. Ideas are full of those products of luxurious thinking: beliefs, conceptions, suppositions, assumptions, what if’s, guesses, and more. Research questions are the articulation, best done in writing, of those ideas that at the least imply a relationship between variables. Why is it best done in writing? Because it is too easy to “get away” with spoken words. It is only when you have to write things down and live with them (spoken words seem to vanish mysteriously) that you face up to what has been said, make a commitment, and work to make sense out of the statement.
Unlike a hypothesis, a research question is not a declarative statement but rather is a clearly stated expression of interest and intent. In the pay-me-now or pay-me-later tradition, the more easily understood and clearer the research question, the easier your statement of a hypothesis and review of the literature will be. Why? From the beginning, a clear idea of what you want to do allows you to make much more efficient use of your time when it comes to searching for references and doing other literature review activities.
Finally, it is time to formulate a hypothesis or a set of hypotheses that reflects the research question. Remember from Chapter 2 the set of five criteria that applies to the statement of any hypothesis? To refresh your memory, here they are again. A well-written hypothesis
· 1. is stated in declarative form
· 2. posits a relationship between variables
· 3. reflects a theory or body of literature upon which it is based
· 4. is brief and to the point
· 5. is testable
When you derive your hypothesis from the research question, you should look to these criteria as a test of whether what you are saying is easily communicated to others and easily understood. Remember, the sources for ideas can be anything from a passage that you read in a novel last night to your own unique and creative thoughts. When you get to the research question stage, however, you need to be more scientific and clearly state what your interest is and what variables you will consider.
Table 3A.1 Research ideas and questions and the hypothesis that reflect them
| Research Interest or Ideas | Research Problem or Questions | Research Hypothesis |
| Open Classroom and Academic Success | What is the effect of open versus traditional classrooms on reading level? | Children who are taught reading in open classroom settings will read at a higher grade level than children who are taught reading in a traditional setting. |
| Television and Consumer Behavior | How does watching television commercials affect the buying behavior of adolescents? | Adolescent boys buy more of the products advertised on television than do adolescent girls. |
| Effectiveness of Checklists in Preventing Hospital Infections | Does the use of checklists when preparing patients for surgery help reduce the level of infection in the hospital? | Those hospitals that regularly use checklists in patient preparation for surgery will have a lower rate of infection per 1,000 patients then these hospitals, which do not. |
| Food Preference and Organic Foods | Do consumers prefer food that i s organ i c? | There will be a difference in preference level as measured by the I ♥ Food scale between those consumers who are offered organic food and those who are offered non-organic food. |
| Use of Energy by Home Owners | Will a home owners’ energy usage change as a function of his or her knowledge of his or her neighbor’s usage? | Those people who know how much energy their neighbors use on a monthly basis will use less energy. |
| Adult Care | How have many adults adjusted to the responsibility of caring for their aged parents? | The number of children who are caring for their parents in the child’s own home has increased over the past 10 years. |
Table 3A.1 lists five research interests, the research questions that were generated from those ideas, and the final hypotheses. These hypotheses are only final in the sense that they more or less fit the five criteria for a well-written hypothesis. Your literature review and more detailed discussion may mean that variables must be further defined and perhaps even that new ones will need to be introduced. A good hypothesis tells what you are going to do, not how you will do it.
TEST YOURSELF
As Pasteur said, chance does favor the prepared mind and you will never know where the best information will come from. So, even if some class seems to contain material unrelated to your specialty or your interests, you never know what insight you might gain from reading widely and discussing ideas with your fellow students. What five things might you read (that you have not) that are related to your interests?
Reviewing the Literature
Here it comes again. Today’s research is built on a foundation of the hard work and dedication of past researchers and their productive efforts. Where does one find the actual results of these efforts? Look to scholarly journals and books and other resources, which are located in the library and online.
The review of literature provides a framework for the research proposal.
Although all stages in the research process are important, a logical and systematic review of the literature often sets the stage for the completion of a successful research proposal and a successful study. Remember one of the fatal mistakes mentioned at the beginning of the chapter about selecting a research question that has been done before? Or one that is trivial? You find out about all these things and more when you see what has already been done and how it has been done. A complete review provides a framework within which you can answer the important question(s) that you pose. A review takes you chronologically through the development of ideas, shows you how some ideas were left by the wayside because of lack of support, and tells you how some were confirmed as being truths. An extensive and complete review of the literature gives you that important perspective to see what has been done and where you are going—crucial to a well-written, well-documented, well-planned report.
So get out your yellow (or recyclable white) writing pads, index cards, pens or pencils, laptop computer, or iPad and let’s get started. Also, don’t forget your school ID card so you can check out books at the campus library.
The literature review process consists of the steps listed in Figure 3A.2 . You begin with as clear an idea as possible about what you want to do, in the form either of a clear and general statement about the variables you want to study or of a research hypothesis. You should end with a well-written, concise document that details the rationale for why you chose the topic you did, how it fits into what has been done before, what needs to be done in the future, and what is its relative importance to the discipline.
There are basically three types of sources that you will consult throughout your review of the literature (see Table 3A.2 ). The first are general sources , which provide clues about the location of references of a general nature on a topic. Such sources certainly have their limitations (which we will get to in a moment), but they can be a real asset because they provide a general overview of, and introduction to, a topic.
Figure 3A.2 The steps in reviewing the literature. It is a formidable task, but when broken down step by step, it is well within your reach.
Table 3A.2 What different sources can do for you in your search for relevant material about an interesting research question
| Information Source | What It Does | Examples |
| General Sources | Provides an overview of a topic and provides leads to where more information can be found | Daily newspaper, news weeklies, popular periodicals and magazines, trade books, Reader’s Guide to Periodical Literature, New York Times Index |
| Secondary Sources | Provides a level of information “Once removed” from the original work | Books on specific subjects and reviews of research |
| Primary Sources | The original reports of the original work or experience | Journals, abstracts and scholarly books, Educational Resources Information Center (ERIC), movies |
For example, let’s say you are interested in the general area of sports psychology but have absolutely no idea where to turn to find more information. You could start with a recent article that appeared in the New York Times (a general source) and find the name of the foremost sports psychologist and then go to more detailed secondary or primary sources to find out more about that person’s work.
The second source type, secondary sources , are “once removed” from the actual research. These include review papers, anthologies of readings, syntheses of other work in the area, textbooks, and encyclopedias.
General, secondary, and primary resources are all important, but very different, parts of the literature review.
Finally, the most important sources are primary sources . These are accounts of the actual research that has been done. They appear as journal articles or as other original works including abstracts. Table 3A.2 summarizes the functions of general, secondary, and primary resources and provides some examples. These three different types of sources are also covered in Chapter 9 in a discussion of historical methods of doing research.
Before you get started, let me share my own particular bias. There is no substitute for using every resource that your library has to offer, and that means spending lots of time turning the pages of books and journals and reading their contents. In many cases, however, there’s no substitute for exploring and using electronic resources such as online databases. You’ll learn about both printed and electronic resources here, but you should remember that you won’t find everything you need online (and much of it is not verifiable), yet online is where the most recent material appears. There is even material now being posted online that will not show up in the library—a new and very interesting development owing to the appearance of online (only) journals and e-books. However, at least for now, begin developing your library as well as your online skills. The online world of literature may someday be the only world of literature, but that surely will not be the case this semester.
One last note before we get started. Your university has an absolute ton of online resources available to you and probably more than you can imagine. How do you find out what might be available? Well, you can access your library online and find out, or follow these steps:
· 1. Go to any of the libraries on your campus.
· 2. Ask for where the reference librarians sit.
· 3. Ask one for a short tour of what’s online (or enroll in one of many classes that most libraries offer at the beginning of the semester to address these skills especially).
One of the best kept secrets on any college campus is how smart and resourceful reference librarians are. Reference librarians are the original search engines. Get to know them (individually)—it will serve you very well.
Using General Sources
General sources of information provide two things: (1) a general introduction to areas in which you might be interested and (2) some clues as to where you should go for the more valuable or useful (in a scientific sense, anyway) information about your topic. They also provide great browsing material.
Any of the references discussed below, especially the indices of national newspapers and so on, can offer you 5, 10, or 50 articles in a specific area. In these articles, you will often find a nice introduction to the subject area and a mention of some of the people doing the research and where they are located. From there, you can look through other reference materials to find out what other work that person has done or even how to contact that person directly.
There are loads of general sources in your college or university library as well as in your local public library and online as well. The following is a brief description of just a few of the most frequently used sources and a listing of several others you might want to consult. Remember to use general sources only to orient yourself to what is out there and to familiarize yourself with the topic.
All of what follows can be accessed online, but the URL (or the Uniform Resource Locator) will differ since you may be accessing it through your university or college.
Readers’ Guide, Full Text Mega Edition is by far the most comprehensive available guide to general literature. Organized by topic, it is published monthly, covering hundreds of journals (such as the New England Journal of Medicine) and periodicals or magazines (such as Scientific American). Because the topics are listed alphabetically, you are sure to find reading sources on a selected topic easily and quickly.
New to the Readers’ Guide world is now the Readers’ Guide Retrospective, which allows access to more than 100 years of coverage from 375 U.S. magazines with indexing of leading magazines back as far as 1890 and citations to more than 3,000,000 articles. If you can’t find something about your interests or related topics here, it’s time to reassess the topic you want to focus on.
Another very valuable source of information is the Facts on File Online Databases with content first published in New York in 1940. Facts on File presents a collection of databases that include tens of thousands of articles and other resources (such as video and audio files) in a multitude of areas. The following list shows you what just some of these databases are and a brief description of each:
· • U.S. Government Online presents in-depth information on the structure and function of the U.S. government.
· • American History Online covers more than 500 years of political, military, social, and cultural history.
· • African-American History Online provides cross-referenced entries, covering African American history.
· • Curriculum Video on Demand provides a video subscription to more than 5,000 educational programs.
· • Science Online contains information on a broad range of scientific disciplines.
· • Ferguson’s Career Guidance Center provides profiles of more than 3,300 jobs and 94 industries.
· • Bloom’s Literary Reference Online contains information on thousands of authors and their works, including an archive of 38,000 characters.
· • And, the grandparent of them all, Facts On File World News Digest, which is the standard resource for information on U.S. and world events
The New York Times Index lists by subject all the articles published in the Times since 1851. Once you find reference to an article that might be of interest, you then go to the stacks and select a copy of the actual issue or view it on microfilm. The originals are seldom available because they are printed on thin paper which was designed to hold up only for the few days that a newspaper might be read.
Instead, contents are recorded on microfilm or some other medium and are available through your library. Many libraries now have microfilm readers that allow you to copy directly from the microfilm image and make a print or hardcopy of what you are viewing. The full text of many newspapers is also now available electronically (discussed later in this chapter). And, although the index is not available online, you can search through the archives of the New York Times online at http://www.nytimes.com —most articles are free to access, but as of this writing, future users are likely to be charged (unless you subscribe to the print edition individually) and of course, probably still free at your local library or institution’s libraries.
Nobody should take what is printed as the absolute truth, but weekly news magazines such as Time(http://www.time.com/time/ ), Newsweek ( http://www.msnbc.msn.com/id/3032542/site/newsweek/ ), and U.S. News and World Report ( http://www.usnews.com/ ) offer general information and keep you well informed about other related events as well. You may not even know that you have an interest in a particular topic (such as ethical questions in research), but a story on that topic might be in a current issue, catch your eye, and before you know it you will be using that information to seek out other sources.
There are two other very comprehensive electronic general source databases: Lexis/Nexis Academic (there are other versions as well) and the Expanded Academic ASAP, both of which are probably available online through your library.
Lexis/Nexis Academic is the premier database. It is absolutely huge in its coverage and contains information on current events, sports, business, economics, law, taxes, and many other areas. It offers full text of selected newspaper articles. Figure 3A.3 shows you the results of a search on the general term “school finance.” You can print this information, e-mail it (to yourself of course if you are in the library and have no other way to record it), and sort in various ways (such as by date).
Figure 3A.3 The results of a simple LexisNexis search.
Copyright © 2007. Reprinted with the permission of LexisNexis, a division of Reed Elsevier Inc. All rights reserved. LexisNexis and the Knowledge Burst logo are registered trademarks of Reed Elsevier Properties Inc. used with the permission of LexisNexis.
The Expanded Academic ASAP is a multidisciplinary database for research, which provides information on disciplines such as the humanities, communication studies, social science, the arts, science, technology and many more disciplines. It covers from 1980 to the present and contains well over 18 million articles.
As the electronic world of resources and reference tools charges long, Google has shown its value in at least two different ways.
First there is Google Scholar, which provides a tool to broadly search for scholarly literature. You can search across disciplines and sources to find articles and books (and other types of publications such as abstracts) and it is the ideal way to locate works by a particular scholar. For example, if you are interested in learning more about what Ron Haskins, a noted expert on policy and families, has done, go to Google Scholar and search; you’ll find works completed by Professor Haskins as well as works in which he is referenced. What a terrific help!
Also, there is Google Books, where Google has undertaken the process of digitizing and making available for no charge millions of books in libraries and other institutions around the world. In Google Books, you can find everything from a limited preview of a book you need for class or the full text of other books that may, or may not, be in the public domain.
Google Books is an absolutely invaluable tool for any researcher, but its use is not without controversy. After all, an author’s work is appearing with no charge to the user and no benefit to the author (such as a royalty payment). The years to come will sort out how tools such as Google Books can be used and still be fair to the author as well as to the researchers.
Then, there is the wealth of information you can dig out of everyday sources such as your local newspaper, company newsletters, and other publications. Thousands of newspapers can be accessed through http://www.newspapers.com , and newspapers often carry the same Associated Press articles as major papers such as the New York Times and the Washington Post. And, please, do not forget the U.S. Government Printing Office (GPO), which regularly publishes thousands of documents on everything from baseball to bees, and the majority of these documents are free—your tax dollars at work. Do you want to know more about the GPO? Visit http://www.gpo.gov for a catalog of what is available.
Finally, there’s the hugely popular and successful Wikipedia (at http://www.wikipedia.org/ ), an encyclopedia that is almost solely based on the contributions of folks like you and me. At this writing, Wikipedia contains over 3,000,000 articles on absolutely everything you can think of. This may be the perfect online place to start your investigations.
Trustworthy? To a great extent, yes. Wikipedia is monitored by content experts, and a recent study found that the venerable Encyclopedia Britannica had more factual errors than did Wikipedia. And, of course, the great thing about any wiki (and it is a general term for anything built on the contributions of many people and open for editing by anyone as well) is that the facts, if incorrect initially, will surely be changed or modified.
The Wikipedia site also contains other wikis, including Wiktionary (a dictionary), Wikinews, Wikiquotes, and more. Just exploring the encyclopedia and these ancillaries is fun.
Finally, one especially useful source that you should not overlook is The Statistical Abstract of the United States, published yearly by the U. S. Department of Commerce ( http://www.census.gov/statab/www ). This national database about the United States includes valuable, easily accessible information on demographics and much more.
Using Secondary Sources
Secondary sources are those that you seek out if you are looking for a scholarly summary of the research that has been done in a particular area or if you are looking for further sources of references.
Major syntheses of information such as reviews can be a terrific foundation for your review.
Reviews and Syntheses of Literature
These are the BIG books you often find in the reference section of the library (not in the stacks). Because so many people want to use them, they must always be available. The following is a summary of some of the most useful. More and more of these books are being published as encyclopedias.
A general secondary source of literature reviews is the Annual Reviews (published since 1930 by Annual Reviews in about 40 disciplines), containing about 20 chapters and focusing on a wide variety of topics such as medicine, anthropology, neuroscience, biomedical engineering, political science, psychology, public health, and sociology. Just think of it—you can go through the past 10 years of these volumes and be very up-to-date on a wide range of general topics. If you happen to find one chapter on exactly what you want to do, you are way ahead of the game. You can find out more about these volumes and see sample tables of contents at http://www.annualreviews.org/ .
Another annual review that is well worth considering is the National Society for the Study of Education(or NSSE) Yearbooks (also available at http://nsse-chicago.org ). Each year since 1900, this society has published a two-volume annual that focuses on a particular topic, such as adolescence, microcomputers in the classroom, gifted and talented children, and classroom management. The area of focus is usually some contemporary topic, and if you are interested in what is being covered, the information can be invaluable to you. The 2009 yearbook has as its focus “localism.”
The Condition of Education in Brief 2007 (available at http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2007066 ) contains a summary of 20 of the 48 indicators in the Condition of Education 2007, including public and private enrollment in elementary/secondary education, the racial/ethnic distribution of public school students, students’ gains in reading and mathematics achievement through third grade, trends in student achievement from the National Assessment of Education Progress in reading and mathematics, international comparisons of mathematics literacy, annual earnings of young adults by education and race/ethnicity, status dropout rates, immediate transition to college, availability of advanced courses in high school, inclusion of students with disabilities in regular classrooms, school violence and safety, faculty salary and total compensation, early development of children, expenditures per student in elementary and secondary education, and public effort to fund postsecondary education. The files are available for downloading.
If you are interested in child development, seek out the Handbook of Child Psychology (Wiley 2006), which is often used as the starting point (for ideas) by developmental and child psychology students, early childhood education students, medical and nursing students, and many others. The four individual volumes are
· • Volume 1: Theoretical Models of Human Development
· • Volume 2: Cognition, Perception and Language
· • Volume 3: Social, Emotional and Personality Development
· • Volume 4: Child Psychology in Practice
Finally, there’s the eight-volume Encyclopedia of Psychology from Oxford University Press (2000), which includes 1,500 articles on every aspect of psychology.
Also, do not forget the large number of scholarly books that sometimes have multiple authors and are edited by one individual or that are written entirely by one person (which, in the latter case, is sometimes considered a primary resource, depending on its content). Use the good old card catalog (or your library’s computerized search system) to find the title or author you need.
Using Primary Sources
Primary sources are the meat and potatoes of the literature review. Although you will get some good ideas and a good deal of information from reading the secondary sources, you have to go to the real thing to get the specific information to support your points and make them stick.
In fact, your best bet is to include mostly primary sources in your literature review, with some secondary sources to help make your case. Do not even think about including general sources. It is not that the information in Redbook or the New Jersey Star Ledger is not useful or valuable. That information is secondhand, however, and you do not want to build an argument based on someone else’s interpretation of an idea or concept.
Get to know your library and where you can find journals related to your field of study. Most libraries offer tours on a regular basis.
Journals
Journals? You want journals? Table 3A.3 lists journals arranged by category. This should be enough for you to answer your professor when he asks, “Who can tell me some of the important journals in your own field?” This list is only a small selection of what is available. The print version of Ulrich’s Periodicals Directory (first published in 1932) lists information on thousands of periodicals, including journals, consumer magazines, and trade publications (at http://www.ulrichsweb.com/ and at your library as well).
Table 3A.3 A sample of the thousands of journals being published in all different fields
| Psychology | |
| AdolescenceAmerican Journal of Family TherapyAmerican Journal of OrthopsychiatryAmerican PsychologistBehavioral DisordersChild DevelopmentChild Study JournalDevelopmental PsychologyContemporary Educational PsychologyEducational and Psychological MeasurementJournal of Abnormal Child PsychologyJournal of Applied Behavioral AnalysisJournal of Autism and Development DisordersJournal of Child Psychology and Psychiatry and Allied DisciplinesJournal of Consulting and Clinical PsychologyJournal of Counseling PsychologyJournal of Educational Psychology | Journal of Experimental Child PsychologyJournal of Experimental Psychology, Human Perception and PerformanceJournal of Experimental Psychology, Learning, Memory, and CognitionJournal of Genetic PsychologyJournal of Humanistic PsychologyJournal of Personality and Social PsychologyJournal of PsychologyJournal of Research in PersonalityJournal of School PsychologyPerceptual and Motor SkillsPsychological BulletinPsychological ReviewPsychology in the SchoolsPsychology of Women QuarterlySmall Group BehaviorTransactional Analysis Journal |
| Special Educational and Exceptional Children | |
| Academic TherapyAmerican Annals of the DeafAmerican Journal of Mental DeficiencyBehavioral DisordersEducation and Training of the Mentally Retarded | Journal of Learning DisabilitiesJournal of Mental Deficiency ResearchJournal of Special EducationJournal of Special Education TechnologyJournal of Speech and Hearing Disorders |
| Education of the Visually HandicappedExceptional ChildrenExceptional Education QuarterlyExceptional ParentGifted Child QuarterlyHearing and Speech ActionInternational Journal for the Education of the BlindJournal for the Education of the GiftedJournal of The Association for the Severely Handicapped | Journal of Speech and Hearing ResearchJournal of Visual Impairment and BlindnessLearning Disability QuarterlyMental RetardationSightsaving ReviewTeaching Exceptional ChildrenTeacher Education and Special EducationTeacher of the BlindTopics in Early Childhood Special Education Volta Review |
| Health and Physical Education | |
| Journal of Health EducationJournal of Alcohol and Drug EducationJournal of Leisure ResearchJournal of Motor LearningJournal of Nutrition EducationJournal of Outdoor EducationJournal of Physical Education, Recreation and Dance | Journal of School HealthJournal of Sport HealthPhysical EducatorResearch Quarterly of the American Alliance for Health, Physical Education, Recreation and DanceSchool Health Review |
| Child Psychology | |
| Adolescence American Journal of Family TherapyAmerican Journal of OrthopsychiatryChild Study JournalContemporary Educational PsychologyDevelopmental PsychologyEducational and Psychological MeasurementJournal of Abnormal Child PsychologyJournal of Applied Behavioral AnalysisJournal of Autism and Development DisordersJournal of Child Psychology and Psychiatry and Allied DisciplinesJournal of Consulting and Clinical PsychologyJournal of Counseling PsychologyJournal of Educational PsychologyJournal of Experimental Child PsychologyJournal of Experimental Psychology, Human Perception and Performance | American Psychologist Behavioral DisordersChild DevelopmentJournal of Experimental Psychology, Learning,Memory, and CognitionJournal of Genetic PsychologyJournal of Humanistic PsychologyJournal of Personality and Social PsychologyJournal of PsychologyJournal of Research in PersonalityJournal of School PsychologyPerceptual and Motor SkillsPsychological BulletinPsychological ReviewPsychology in the SchoolsPsychology of Women QuarterlySmall Group BehaviorTransactional Analysis Journal |
| Special Education and Exceptional Children | |
| Academic TherapyAmerican Annals of the DeafAmerican Journal of Mental DeficiencyBehavioral DisordersEducation and Training of the Mentally RetardedEducation of the Visually HandicappedExceptional ChildrenExceptional Education QuarterlyExceptional ParentGifted Child Quarterly | Journal of Learning DisabilitiesJournal of Mental Deficiency ResearchJournal of Special EducationJournal of Special Education TechnologyJournal of Speech and Hearing DisordersJournal of Speech and Hearing ResearchJournal of Visual Impairment and BlindnessLearning Disability QuarterlyMental Retardation Sightsaving Review |
| Hearing and Speech ActionInternational Journal for the Education of the BlindJournal for the Education of the GiftedJournal of The Association for the SeverelyHandicapped | Teaching Exceptional ChildrenTeacher Education and Special EducationTeacher of the BlindTopics in Early Childhood Special EducationVolta Review |
| Health and Physical Education | |
| Journal of Alcohol and Drug EducationJournal of Health EducationJournal of Leisure ResearchJournal of Motor LearningJournal of Nutrition EducationJournal of Outdoor EducationJournal of Physical Education, Recreation and DanceJournal of School HealthJournal of Sport HealthLibrary ResearchLifelong Learning:The Adult YearsMathematics and Computer EducationMathematics TeacherModern Language JournalMusic Education JournalNational Education Association Research BulletinStudies in Art EducationStudies in Educational EvaluationTeachers College RecordTheory and Research in School Education | National Elementary PrincipalNegro Education ReviewPeabody Journal of EducationPhi Delta KappanPhysical EducatorResearch Quarterly of the American Alliance for Health, Physical Education, Recreation and DanceReview of Educational ResearchSchool Health ReviewSchool Library Media QuarterlySchool Psychology ReviewSchool Science ReviewScience and ChildrenScience EducationScience TeacherSecondary School Theatre JournalSocial EducationTheory Into PracticeToday’s EducationVoc EdYoung Children |
| Sociology and Anthropology | |
| American AnthropologistAmerican Behavioral ScientistAmerican Journal of SociologyAmerican Sociological ReviewAnthropology and Education QuarterlyChild WelfareFamily RelationsGroup and Organization StudiesHuman OrganizationHuman Services in the Rural EnvironmentJournal of Correctional Education | Journal of Marriage and the FamilyRural SociologySex Roles: A Journal of ResearchSocial WorkSociology and Social ResearchSociology of EducationUrban AnthropologyUrban EducationUrban ReviewYouth and Society |
| Analytical Research | |
| Administration and SocietyAmerican Historical ReviewAmerican Political Science ReviewAnnals of the American Academy of Political andSocial ScienceCivil Liberties LawComparative Education Review | DaedalusEconomics of Education ReviewEducation and Urban SocietyEducation ForumEducational StudiesEducational TheoryHarvard Civil Rights |
| Nursing | |
| AACN Advanced Critical CareAdvanced Emergency Nursing JournalAdvances in Neonatal CareAdvances in Nursing Science – Featured JournalAdvances in Skin & Wound Care:The Journal forPrevention and HealingAJN, American Journal of NursingAlzheimer’s Care TodayCancer NursingCIN: Computers, Informatics, NursingCritical Care Nursing QuarterlyDimensions of Critical Care NursingFamily & Community HealthGastroenterology NursingHealth Care Food & Nutrition FocusHealth Care Management ReviewThe Health Care ManagerHolistic Nursing PracticeHome Healthcare NurseInfants & Young ChildrenJournal for Nurses in Staff DevelopmentJournal of Ambulatory Care ManagementJournal of Cardiovascular NursingJournal of Christian NursingJournal of Head Trauma RehabilitationJournal of Hospice and Palliative Nursing | Journal of Infusion NursingJournal of Neuroscience NursingJournal of Nursing Care QualityThe Journal of Nursing ResearchJournal of Perinatal and Neonatal NursingJournal of Public Health Management & PracticeJournal of the Dermatology Nurses’ AssociationJournal of Trauma NursingMCN, The American Journal of Maternal/ChildNursingMen in NursingNurse EducatorNursing 2010Nursing 2010 Critical CareNursing Administration QuarterlyNursing Made Incredibly Easy!Nursing ManagementNursing ResearchNutrition TodayOncology TimesOR Nurse 2010Orthopaedic NursingOutcomes ManagementPlastic Surgical NursingProfessional Case ManagementQuality Management in Health Care |
Journals are by far the most important and valuable primary sources of information about a topic because they represent the most direct link among the researcher, the work of other researchers, and your own interests.
What actually is a journal, and how do papers or manuscripts appear? A journal is a collection (most often) of research articles published in a particular discipline. For example, the American Educational Research Association (AERA) publishes more than six journals, all of which deal with the general area of research in education. The American Psychological Association (APA) publishes many journals including the Journal of Experimental Psychology and the Journal of Counseling Psychology. The Society for Research in Child Development (SRCD) publishes Child Development and Child Development Monographs,among others. Membership in these professional groups entitles you to a subscription to the journals as part of the package, or you can subscribe separately.
Most often, these professional organizations do not do the actual publishing themselves, but only the editorial work where the manuscripts are reviewed and considered for publication. For example, Child Development, sponsored by the SRCD, is published by Wiley Publishers/ Blackwell.
How do most respectable journals work? First, a researcher writes an article within the province of the particular journal to which it is being submitted. The manuscript is prepared according to a specific format (such as the one shown in Chapter 14), and then usually three copies are submitted to the journal editor. Guidelines for preparing manuscripts are usually found on the front or back covers of most journals in the social and behavioral sciences. Often the journal requires that the author follow guidelines stated in the sixth edition of the American Psychological Association Publication Manual ( 2009 ).
The peer review process of reviewing journal submissions ensures that experts review and comment on a research manuscript before it is published.
Second, once the article has been received by the editor, who is an acknowledged expert in that particular field, the article is sent to at least three reviewers who are also experts in the field. Note that today, almost all of the submission and review process occurs online. These reviewers participate in a process known as peer review, in which the reviewers do not know the identity of the author (or authors) of the paper. The author’s name appears only on a cover sheet, which is removed by the editor. A social security number, or some other coded number, is used for identification on the rest of the manuscript. This makes the process quite fair (what is called “blind”)—the reviewer’s chance of knowing the identity of the author is greatly reduced, if not eliminated. The possibility that personalities might get in the way of what can be a highly competitive goal—publishing in the best journals—is thus minimized. Each reviewer makes a recommendation regarding suitability for publication. The options from which the reviewers select can include
· • Accept outright, meaning that the article is outstanding and can be accepted for publication as is
· • Accept with revisions, meaning that some changes need to be made by the author(s) before it is accepted (and is of course reviewed again)
· • Reject with suggestions for revisions, meaning that the article is not acceptable as is, but after changes are made the author(s) should be invited to resubmit it
· • Reject outright, meaning that the article is completely unacceptable and is not welcome for resubmission
Finally, when a consensus is reached by the reviewers, the editor of the journal conveys that decision to the author(s). If a consensus cannot be reached, the editor makes a decision or sends the article to another reviewer for additional comments. Editors work very hard to ensure that the review process and the journal publication process are fair.
By the way, you might be interested to know that the average rejection rate for the top journals is about 80%. Yes, 80% of the articles submitted never get in, but those rejected by the top journals usually find their way into other journals. Just because these articles are not accepted by the journals with the highest rejection rate does not mean they cannot make a significant contribution to the field. In fact, several studies have shown that there is little consistency among reviewers, and what one might rank high, another might rank quite low. However, in general, it’s safe to say that the better scientific reports are published by the better journals.
One more note about primary sources in general. If you know of a journal or a book that you might need and your library does not have it (and it is not available online), do not despair. First, check other libraries within driving distance or check with some of the professors in your department. They might have it available for loan. If all else fails, use the interlibrary loan system, with which your reference librarian will be glad to help you. This service helps you locate and physically secure the reference materials you want for a limited amount of time from another library. The system usually works quickly and is efficient.
Abstracts
If journals are the workhorses of the literature review, then collections of abstracts cannot be very far behind with regard to their convenience and usefulness. An abstract is a one- (or at most two-) paragraph summary of a journal article which contains all the information readers should need to decide whether to read the entire journal article.
Abstracts help you save the time it would take to locate potentially important sources of information.
By perusing collections of abstracts, researchers can save a significant amount of time compared with leafing through the journals from which these abstracts are drawn. Most abstracts also include subject and author indexes to help readers find what they are looking for, and abstracts of articles routinely appear in more than one abstract resource.
For example, a study on the benefits of long-distance learning might appear in PsychINFO from the International Journal of Simulation and Process Modelling.
The following is a brief description of some abstract collections you might find useful.
One well-known collection of abstracts is PsycINFO (at http://www.apa.org/pubs/databases/psycinfo/index.aspx ). PsycINFO (for members of APA) and PsycINFO Direct (for nonmembers) provide an electronic database that contains abstracts and summaries of psychological literature from the 1800s to the present. Some facts about PsycINFO: It contains articles and abstracts from more than 2,500 journals, is updated weekly, offers chapters from scholarly books, contains material from 49 different countries, covers dissertations, and much more. No doubt—on your research travels, it is a great resource.
There is an unlimited amount of information in PsycINFO, and the online nature enables you to search electronically. Figure 3A.4 shows you a sample PsycINFO screen for a journal article. Screens for books and chapters and dissertations look quite similar.
One other way to use PsycINFO is to look up the key word “bibliography.” Under this heading, you will find a list of bibliographies that have already been published. You might be lucky and find one that focuses on your area of interest.
One index that is especially useful is Educational Resources Information Center, or ERIC. ERIC ( http://www.eric.ed.gov/ ) is a nationwide information network that acquires, catalogs, summarizes, and provides access to education information from all sources. It currently contains more than 1.3 million education-related documents and adds about 30,000 per year. The database and ERIC document collections are housed in about 3,000 locations worldwide, including most major public and university library systems.
Figure 3A.4 The results of a PsycINFO search. Screenshot is reprinted with permission of the American Psychological Association, publisher of the PsycINFO database, All rights reserved.
ERIC produces a variety of publications and provides extensive user assistance with several different ways to search the database. As with PsycINFO, the ERIC system works with a set of descriptive terms found in a thesaurus, the Thesaurus of ERIC Descriptors (see Figure 3A.5 ), which should be your first stop. Once you find the search words or descriptors, you can use the subject index (published monthly) until you find the number of a reference focusing on what you want. Finally, you are off to the actual description of the reference, as you see in Figure 3A.6 . Most of the time, these ERIC documents are in PDF (portable document format) and you can access the entire document. Other times, although rare, you may have to order directly from the ERIC clearinghouse. If your library has a government documents department, it might already have the document on hand. Also, you might be able to contact the original author as listed in the résumé.
ERIC has been in business since 1966 and has regional clearinghouses that archive, abstract, and disseminate educational articles and documents. Education is broadly defined, so many disciplines in the social and behavioral sciences are covered quite adequately.
Do you think that this is enough to get started? PsycINFO and the ERIC sets of abstracts are major resources, but there are others that are a bit more specialized and also very useful.
Figure 3A.5 The set of ERIC terms in the thesaurus you start with when conducting an ERIC search.
Figure 3A.6 Once you have identified areas through the ERIC thesaurus, it’s time to turn to key words that produce ERIC entries like these.
Titles of other abstracts, such as Sociological Abstracts, Exceptional Child Education Resources, Research Related to Children, and Dissertation Abstracts, reveal the wide variety of available reference material.
Finally, there’s ProQuest Dissertations and Theses (which replaces Dissertation Abstracts at many libraries) at http://www.proquest.com/en-US/catalogs/databases/detail/pqdt.shtml , which contains over 2.7 million dissertations and theses dating from 1861, with full text online from 1997. More than 75,000 new entries are added every year and it contains the abstracts of over 2,000,000 dissertations from 1861 to the present, in the following areas:
· • Agriculture
· • Astronomy
· • Biological and Environmental Sciences
· • Business and Economics
· • Chemistry
· • Education
· • Engineering
· • Fine Arts and Music
· • Geography and Regional Planning
· • Geology
· • Health Sciences
· • History and Political Science
· • Language and Literature
· • Library and Information Science
· • Mathematics and Statistics
· • Philosophy and Religion
· • Physics
· • Psychology and Sociology
Indices
Journals and abstracts provide the substance of an article, a conference presentation, or a report. If you want a quick overview of where things might be located, turn to an index, which is an alphabetical listing of entries by topic, author, or both.
The widely used and popular Social Sciences Citation Index (SSCI) and Science Citation Index (SCI) work in an interesting and creative way. SSCI (at http://thomson-reuters.com/products_services/science/science_products/a-z/social_sciences_citation_index ) provides access to bibliographic information, author abstracts, and citations from more than 2,400 journals in more than 50 disciplines. SCI (now part of the Web of Science at http://thomsonreuters.com/products_services/science/science_products/a-z/science_citation_index ) provides researchers access to over 3,700 scientific and technical journals across 100 disciplines.
Indices help you locate the sources of important information.
Let’s say you read an article that you find to be very relevant to your research proposal and want to know what else the author has done. You might want to search by subject through abstracts, as we have talked about, but you might also want to find other articles by the same author or on the same general topic. Tools like SSCI and SCI allow you to focus on your specific topic and access as much of the available information as possible. For example, do you want to find out who has mentioned the classic article “Mental and Physical Traits of a Thousand Gifted Children,” written by Louis Terman and published in 1925? Look up Terman, L., in SSCI year by year, and you will find more references than you may know what to do with.
Finally, you can consult the Bibliographic Index Plus online (at http://www.hwwilson.com/Databases/biblio.htm ), a compilation of bibliographies that results from a search of more than 530,000 bibliographies. Just think of the time you can save if you locate a relatively recent bibliography on what interests you.
TEST YOURSELF
What is the best use to which you can put a general, secondary, and primary source and name one of each which you might use in better understanding the most important questions in your own field of study?
Reading and Evaluating Research
Almost any research activity that you participate in involves reading research articles that appear in journals and textbooks. In fact, one of the most common faults of beginning researchers is not being sufficiently familiar with the wealth of research reports available in their specific area of interest. It is indeed rare to find a research topic about which nothing (or nothing related) has been done. You may not be able to find something that addresses the exact topic you wish to pursue (such as changes in adolescent behavior in Australian children who live in the outback), but there is plenty of information on adolescent behavior and plenty on children who live in Australia. Part of your job as a good scientist is to make the argument why these factors might be important to study.
You can do that by reading and evaluating research that has been done in various disciplines on the same topic.
Research articles and reports must always be carefully evaluated and the results never taken at face value.
What Does a Research Article Look Like?
The only way to gain expertise in understanding the results of research studies is to read and practice understanding what they mean. Begin with one of the journals in your own area. If you don’t know of any, do one of two things:
· • Visit your adviser or some faculty member in the area in which you are interested and ask the question, “What are the best research journals in my area?”
· • Visit the library and look through the index of periodicals or search online some of the resources we just identified. You are bound to find hundreds of journals, most online.
For example, for those of you interested in education and psychology and related areas, the following is a sample of 10 research journals that would be a great place for you to start:
· • American Educational Research Journal
· • American Psychologist
· • Educational Researcher
· • Educational and Psychological Measurement
· • Harvard Educational Review
· • Journal of Educational Research
· • Journal of Educational Psychology
· • Journal of Educational Measurement
· • Phi Delta Kappan
· • Review of Educational Research
Here are 10 more that focus primarily on psychology:
· • Child Development
· • Cognition
· • Human Development
· • Journal of Applied Developmental Psychology
· • Journal of Experimental Psychology
· • Journal of Personality and Social Psychology
· • Journal of School Psychology
· • Perceptual and Motor Skills
· • Psychological Bulletin
· • Sex Roles
And, don’t forget our previous discussion of Ulrich’s periodical guide (over 300,000 entries).
Criteria for Judging a Research Study
Judging anyone else’s work is never an easy task. A good place to start might be the following checklist, which is organized to help you focus on the most important characteristics of any journal article. These eight areas can give you a good start in better understanding the general format of such a report and how well the author(s) communicated to you what was done, why it was done, how it was done, and what it all means.
Research articles take all kind of shapes and forms, but their primary purpose is to inform and educate the reader.
· 1. Review of Previous Research
· • How closely is the literature cited in the study related to previous literature?
· • Is the review recent?
· • Are there any seminal or outstanding references you know of that were left out?
· 2. Problem and Purpose
· • Can you understand the statement of the problem?
· • Is the purpose of the study clearly stated?
· • Does the purpose seem to be tied to the literature that is reviewed?
· • Is the objective of the study clearly stated?
· • Is there a conceptual rationale to which the hypotheses are grounded?
· • Is there a rationale for why the study is an important one to do?
· 3. Hypothesis
· • Are the research hypotheses clearly and explicitly stated?
· • Do the hypotheses state a clear association between variables?
· • Are the hypotheses grounded in theory or in a review and presentation of relevant literature?
· • Can the hypotheses be tested?
· 4. Method
· • Are both the independent and dependent variables clearly defined?
· • Are the definitions and descriptions of the variables complete?
· • Is it clear how the study was conducted?
· 5. Sample
· • Was the sample selected in such a way that you think it is representative of the population?
· • Is it clear where the sample came from and how it was selected?
· • How similar are the participants in the study to those who have been used in similar studies?
· 6. Results and Discussion
· • Does the author relate the results to the review of literature?
· • Are the results related to the hypothesis? Is the discussion of the results consistent with the actual results?
· • Does the discussion provide closure to the initial hypothesis presented by the author?
· 7. References
· • Is the list of references current?
· • Are they consistent in their format? Are the references complete?
· • Does the list of references reflect some of the most important reference sources in the field?
· 8. General Comments About the Report
· • Is the report clearly written and understandable?
· • Is the language biased?
· • What are the strengths and weaknesses of the research?
· • What are the primary implications of the research?
· • What would you do to improve the research?
· • Does the submitted manuscript conform to the editor’s or publisher’s specifications?
Using Electronic Tools in Your Research Activities
Imagine this if you will: You are in your apartment and it is late at night. You find that you need one more citation on the development of charter schools to complete your literature review. You are tired. It is snowing. The library is about to close, and it might not have what you need anyway.
Zoom, you’re on the Internet and you’re on the way. Log on to your library and access one of their many databases to search for the information you need. In 20 seconds you have the reference to read or print. Is this for real? You bet, and since the printing of the last edition of Exploring Research (some staggering 3 years ago), online tools and databases are even more dominant forces in preparing, conducting, and disseminating research.
Both the computer as a tool and the library as a storehouse of information play different, but equally important and complementary, roles in the research process.
Whether at home, in your office, or in the confines of the library—and now using wireless technology at the mall or in front of the student union—the use of technology for completing literature searches and reviews is booming, and blooming with new databases to search becoming available each day.
In a moment we’ll start our explanation of some of this, but first a few words of “this can’t be true, but it is.” Many of you who are using this book may have never taken advantage of what your library services have to offer. You may not, for whatever reason, access these from off campus, but what is not understandable is why you are not accessing these resources on campus. All colleges and universities (and, of course, the local public library) provide free access to all these resources for students. The personal computers you can use may be located in the computer center, in the library, in academic buildings, or even in all three and more—but they are surely there for the using. It is likely that a hefty chunk of the fees that you pay each semester goes toward purchasing new equipment and paying for these services, so use them!
And just a few more words about libraries in general. We all know how easy it is to explore a library’s contents online—it’s quick, easy, and usually very reliable. But, there is also a huge benefit to actually physically visiting the library other than to take the orientation workshop we mentioned earlier in the book. Here’s the thing: What you may find in the library, incidental to what you are looking for, you may never find online. For example, you’re in the stocks exploring articles on charter schools and reading through journal articles organized by volume. Aren’t you delightfully surprised to find that the article before the one you are looking for seems to contain some very relevant information to the question you are asking? And, you take out a few more volumes, find a nice easy chair, turn off your MP3 player, and find even more—treasures that were unanticipated, but nonetheless, very valuable. Make a visit—you’ll be delightfully surprised.
Searching Online
At the University of Kansas, students can walk into Watson Library (one of the main research libraries), sit down at a computer terminal, access ERIC documents, and search through them in seconds for the references of interest—not bad. They can access a network connection that can lead them to millions of other abstracts and full-length articles from hundreds of databases “leased” by the university each year. And they can, of course, do all this from the comfort of their dorm room, apartment, or home 10 or 1,000 miles away. In fact, if they have any difficulty during their online activity, they can even Ask the Librarian—that’s right, open a new window in the browser and enter a question such as “Does the New York Timesstill have an index?” or “What is the leading journal on business education?” These reference librarians are not known as the original search engine for no good reason. They know lots, but most importantly, they know where to find the answers—the key to a good research foundation.
University, business, and government researchers are turning to online information providers more and more to find the key information they need, whether a specific reference or fact, such as the number of bicycles manufactured by Japan or the number of young adults who live in urban areas.
Your local public library, as well as the university’s library system, has access to the Internet as well as guides to the information available electronically.
The Value of Online Searches
Doing online searches boils down to a savings of time and convenience and in some cases, thoroughness versus a visit to the library. You can do a search using one of the online services in a quarter of the time it takes to do it manually.
Another important advantage of online searches, if your search skills are anywhere near competent, is that you are not likely to miss very much. The information providers provide access to tens of thousands of documents, either in their own databases or in others they can access. Dedicated databases have millions of pieces (such as the APA’s PsycINFO) of information. Most colleges and universities now allow access to their libraries from off campus, and an increasing number allow you access to the complete record of the article (as a PDF), not just an abstract.
Finally, and this may be the most attractive advantage, online searches are the way of the future. There is so much information out there that soon it will be close to impossible to search intelligently without the aid of a computer.
If there is any real downside (as we mentioned earlier), it’s that when you use online services, you don’t get a chance to browse among the thousands of books at the library and since books are organized by area of specialization you will very often find yourself opening books that you didn’t even know existed and finding things that can be very valuable.
The Great Search Engines
Although there is no central listing of Web sites, there are search engines that can help you find what you are interested in. For example, the most popular search engine, by far, is Google ( www.google.com ), and more about that soon. Fill in the term you are looking for and click Google Search and you are bound to find material you can use. Better yet, combine words such as “résumé nursing” to find people who have entered that phrase on their résumé. Type in “ www.yahoo.com ,” which takes you to an opening page with hundreds of links to topics in every area imaginable.
For example, let’s say you are interested in finding information on homelessness. As you can see in Figure 3A.7 , almost 7,000,000 results came up in less than .3 of a second. Amazing. Figure 3A.7 shows the term entered in the search area of Google and the results of that search. We’ll get to an analysis of a Google screen later in this section.
Search engines are tools that help you sift through the thousands of pages of information available on the Internet and identify the specifics of what you need.
Figure 3A.7 A sample Google search.
Google™ is a registered trademark of Google, Inc.
After the search is completed, the results will show several suggested links which you then can click on to find out the contents of the home pages that were found.
Are all search engines created equally? No. And one of the ways in which they are not created equal is what they are best suited for. Table 3A.4 lists a variety of search engines by what they do. The URL don’t have the ubiquitous http://www as the start of each one since browsers such as Firefox, Internet Explorer, Chrome, and Bing can search and locate with that additional information (and keystrokes on your part).
You can also consult a search engine that, in itself, searches many different search engines. For example, search engines such as SurfWax ( www.surfwax.com ) and Mamma ( www.mamma.com and billed, of course, “The Mother of All Search Engines®), are meta-search engines, or those search engines that return the results of exploring many search engines all at once. Let’s say that your research involves looking at the history of baseball and you need to review various major league teams. In Figure 3A.8 you can see the results of a SurfWax search for information about the Washington Nationals where almost 70,000,000 pages were identified.
Here are some tips about using a search engine:
· • Enter the narrowest search terms and then broaden your search from there. Entering “intelligence” will find lots of stuff, most of it irrelevant; however, if you enter “intelligence” and “children” and “school,” the results will be much more manageable and closer to what you want. Remember that the fewer the words you enter, the more general the results will be.
· • If you use more than one word, join them with the conjunction “AND,” such as bilingual AND education, or use quotes, such as “bilingual education.” This is the default for some search engines but not all.
· • If a help file or function comes along with the search engine, open it and read it. It will have invaluable information that will save you time and effort.
· • When you become more accustomed to using a search engine, look for the more advanced searching techniques and use them.
· • Didn’t you get what you wanted? The simplest solution is to check your typing. Simple typos spell disaster.
The original, and still the best, search engine is your reference librarian who never crashes, is always available, tends to be helpful, and is very knowledgeable.
Table 3A.4 Different types of search engines and what they search for
| If you need to do a general, all purpose search . . . | |
| Name | URL |
| Alltheweb | google.com |
| AltaVista | altavista.com |
| AOL | aol.com |
| ASK | ask.com |
| Cull | cull.com |
| Gigablast | gigablast.com |
| google.com | |
| Lycos | lycos.com |
| MSN | msn.com |
| Yahoo | Yahoo.com |
| If you want to search for blogs about a particular topic . . . | |
| Name | URL |
| Blogpulse | Blogpulse.com |
| Google blog search | blogsearch.google.com |
| LJSeek | Ljseek.com |
| Opinmind | Opinmind.com |
| Tecnhorati | Technorati.com |
| If you want to search for books . . . | |
| Name | URL |
| Your local library! | Easy to find at your school URL |
| Amazon | Amazon.com |
| BookFinder | Bookfinder.com |
| Google Scholar | scholar.google.com |
| Google Books | Books.google.com |
| ReadPrint | ReadPrint.com |
| If you want to search for images . . . | |
| Name | URL |
| Picsearch | Picsearch.com |
| PhotoBucket | photobucket.com |
| New York Public Library Digital Gallery digitalgallery.nypl.org | |
| Classroom Clipart | classroomclipart.com |
| stock .xchng | www.sxc.hu |
Courtesy of Google Inc.
· • Try a synonym for the term or terms you’re looking for. There’s more than one way to eviscerate a feline (get it?).
And if you want to find out even more about search engines, go to websearch.about.com and Wendy Boswell’s all-informative and useful About.com search engine site.
More About Google
Although Google is the most popular search engine and its share of searches continues, and you may use it regularly, it is still worth exploring what it does and how it does it. It regularly catalogues millions of web pages and returns results in very short order. Since it is so popular, here are some specific tips about using this search engine, including some special features you may not know about.
Not just Google, but every search engine has its own special tips and tricks you can learn (at their Web site) to facilitate your searching activities and increase your success rate.
Figure 3A.8 Searching for information about the Washington Nationals Baseball team.
Google™ is a registered trademark of Google Inc.
Google Search Results
Figure 3A.9 shows a search conducted on the term “grade retention.” There’s more to the search results than meets the eye (not only a listing of other Web sites), and here’s a more detailed analysis on what’s in that window and how it might help you.
· 1. Across the top of the Google search results is a listing of other “tabs” you can click on to find additional information about the topic (Web, Images, Videos, Maps, News, etc.). For example, if you want to find news about the topic on which you searched, click on News. In this case, you can find related news stories that can further your understanding of this topic.
· 2. To the right of the Google search area (where you enter the terms for which you want to search) are Advanced search and Preferences options. These basically allow you to refine your searches and are easy to learn on your own but surely not necessary as you are learning to use Google and even when you are a fairly competent Google user. As we said earlier, the more refined the words you identify as search terms, the better your results will be.
· 3. In this example, there are no sponsored links (really advertisements on which Google makes a ton of money), which are usually located on the right-hand side of the page. These advertisements are located away from the results listing so that you very clearly know they are to be treated separately.
· 4. Below (and to the right of) the Google search term (in this case “grade retention”) is a tally of the results, showing that 1,460,000 “hits” accumulated in .23 second (fast!). Note that if you repeated the same search, you will get a different outcome (probably just slightly) since things change so fast.
Figure 3A.9 A Google search on the phrase “grade retention” and the results.
· 5. Right below the results line is the all-important results of the search. Most show the following:
· a. The title of the page (Grade Retention—The Great Debate). Notice how the words “grade retention” are highlighted since this is one of the original search words.
· b. Next is a brief abstract of the contents of that page, which should allow you to determine whether it is worth exploring.
· c. Next is the URL, or the Web address, for this particular page followed by the size of the page, the cache (any stored record of this page), and other pages that are similar to this one. As always, you can click on any underlined link.
Word Order and Repetition
You already know that word order matters (we talked about that earlier), but the repetition of words in the search box matters as well.
For example, you saw in Figure 3A.8 the result of a search on grade retention. However, if we enter the search terms “grade retention retention” (we entered it twice), then the weighting of the search leans more toward retention, less toward grade. Similarly, if we entered the terms “grade grade retention,” the search would be weighted toward the topic of grades. Word repetition is not a science, but it does allow you to prompt Google to provide another set of results on the same topic.
Using the Phonebook
This may be the greatest nondocumented, and not generally known, tip and feature about using Google.
A great deal of what we all do as researchers is to find information and locate people. If you find a particularly interesting research article and want to know more about the topic, there’s just nothing wrong with searching for more information about the author of that article and contacting him and her.
For example, let’s say you want to contact this author. The first place to try is his home institution (the University for Kansas, which you can find at www.ku.edu ). This should get you what you want. Let’s say, however, that in spite of your efforts, you have no luck.
Using the Google phonebook feature, you can enter the terms phonebook:salkind ks (notice there is no space after the colon and you have to know the state in which the listing is located), and you’ll get the contact information you need. You can reverse the process as well by entering the phone number and seeing the listing. For your information, rphonebook will search only for residential listing and bphonebook only for business listings.
Using Bibliographic Database Programs
Anyone who does research and writes about that research can tell you that one of the most tedious parts of writing a research manuscript is references, references, references—keeping track of them, entering them, and organizing them is just about the least fun anyone can have.
There are a welcome set of tools that can help you do these three things and more. Bibliographic database programs are tools that help you manage your set of references, and the best ones allow you to do things such as
· • Enter the data for any one reference using a standard form
· • Change the format to fit the manuscript requirements, such as the American Psychological Association (APA) or the Modern Language Association (MLA)
· • Search the database of references using key words
· • Add notes to any one reference which can also be searched
· • Generate a final list of references for use in the manuscript
Although one of the most tedious, time-consuming parts of creating a research document is tracking and dealing with bibliographic references, there are now several different software programs that can greatly reduce the necessary time and effort.
You can, of course, do all these by using 3″ × 5″ index cards, but entering the references only once and never having to retype them, track them, and organize them—we could go on and on, but we think you get the picture.
A bunch of such bibliographic database programs are available—some of them free and some of them commercially available. Let’s take a look at EndNote ( http://www.endnote.com ), a commercially available product. All of these tend to offer the same features—you enter information about the reference, and the tool formats it according to the format you specify. They all accomplish this goal in different ways and also offer different bells and whistles, so you should take advantage of the free download and try them out. Other commercial products that work well are ProCite ( www.procite.com ) and Biblioscape (http://www.biblioscape.com ). Be sure that the program works on your operating system because some only work for a Windows- or a Mac-based operating system.
As you can see in Figure 3A.10 , EndNote works by your choice of the type of references (book, journal, web page) and then entering the pertinent information. The information then appears in your “library” (we created on named “term paper”). Once finished creating the library of references, EndNote (or another application) generates the bibliography for you with a few clicks, formatted as you want or even using a custom format.
As you can see, each element of the reference (author, date, etc.) is entered in its own space. You complete a separate form for each reference (be it a journal article, a book chapter, or a presentation at a convention) and you select the entry format.
Figure 3A.10 Using Endnotes, one of many citation creation tools.
Courtesy of Google Inc.
Looking for Articles Online
This clever design from the Google People fits very well the needs of any researcher, from the most basic to the most advanced.
Researchers are in the business of finding information and using that information to lay the groundwork for their research. One might search specific sites such as the Washington Post, U.S. News & World Report, or the American Psychological Association, and one would surely find material about a particular topic. But Google is an excellent tool for finding information across many different sites since it will look not only for topics that may have appeared on a particular site, but also for topics that appear secondarily to that site. For example, a search on the NYT Web site for articles on day care would result in a bunch of productive hits. But, how about a search for articles on this topic that may have appeared originally in the Times but in other locations as well? Of course, this can be done for newspapers, periodicals, magazines, journals—anywhere material might appear. How to do it?
Here are the search terms for a simple search for articles about day care in the New York Times: day care site: www.nytimes.com .
Day care appears in quotes so Google will look for it as a set of terms and not just “day” and then “care.” This search results in 28,100 hits.
Now, if we search for the magic words copyright * The New York Times Company day care, we find 851 hits, which includes all the articles on day care from the Times, as well as all the articles used by other publications from the Times (in which they may have cited the Times).
The * in the search terms acts as a wild card so any year of copyright is searched for, and we could get rid of the site: command since the New York Times Company (which is their copyright line) serves the same purpose. Pretty cool.
Finding Tons of Directories and Lists
This is the last Google tip, but another one that could prove invaluable. Much of our job as researchers is to find information, but also collections of information. The command intitle: can serve us quite well.
For example, the search terms intitle: directory day care would return listings of directories containing information about day care. If we changed the search terms to include a wild card, such as intitle: directory * day care, we then get a much more broadly defined list since it can include elderly day care, adult day care, Miami day care, and so on—and the number of returns is much, much higher than the simple direct search we first showed you.
More About Google Than You Can Imagine
Google has a set of help centers located at http://www.google.com/support/ where you should go if you need support about one of their products such as Gmail, Google Docs, or help on searching the Web.
Advanced Google Search Tools
Sure it’s easy to find the phone number of a researcher who lives in Wyoming, but phonebook is only one of many search operators that Google allows to help you refine what you want to do and you can find about all of them at http://www.googleguide.com/advanced_operators.html .
For example, you can use the search operator “define:” to find the definition of a word. So, entering define:mysticism will give you huge lists of the definition of the term on various locations around the Web. If you typed in “definition of mysticism”, you would get web pages that define mysticism, but not central directory of definition. Another really useful operator is “source”, which provides a search on a particular topic limited to the source you identify. For example, if you want to search for information about iPads, but only which appeared in the New York Times, iPad source:New York Times would provide you with a nice collection of articles that have appeared. You can even search for the latest weather report (weather:losangeles) and yes, what time the movies are showing (movie:title such as movie:Greenberg Lawrence, ks) and it really works!
TEST YOURSELF
It’s really easy—and maybe too easy—to conduct your background research online without regard to that massive building in the idle of campus called the library. Do you think it is adequate to conduct your literature review online? What advantages does this strategy offer? Disadvantages?
Using the Internet: Beyond Searches
Most of you who are reading this text are very savvy when it comes to using the Internet, but there are still some of you who are not. The following material is a refresher for those who can always learn something new and an introduction to those who are unfamiliar with the Internet, how it works, and what it can do for a new researcher.
In the most basic of terms, the Internet is a network of networks. A network is a collection of computers that are connected to one another and can communicate with each other. Imagine all these networks being connected to one another and imagine hundreds of networks and thousands of computers of all different types attached to one another and millions of people using those computers. Now you have some idea how large the Internet is. It is growing geometrically and millions of people connect every day for work, for fun, and of course, to pursue research activities.
Research Activities and the Internet
If you are talking about information in all shapes and sizes, there is not much that you cannot do on the Internet. Here is a brief overview of how the Internet can be used for research purposes:
· • The Internet is often used for electronic mail or e-mail . You can exchange postal mail with a colleague across the United States or the world, but you can also do the same without ever putting pen to paper. You create a message and send it to your correspondent’s electronic address with documents, images, and more attached. It is fast, easy, and fun. For example, if you would like a reprint of an article you find interesting, you could e-mail the author and ask for a copy and it may very well come back to you electronically. Virtually all faculty, staff, and students at educational institutions have access to e-mail. Also if you want further information about a particular person’s work, you could probably find his or her résumé online.
· • Thousands of electronic newsgroups , often called Usenet newsgroups, are available on the Internet. These are places where information can be posted and shared among Internet users, with topics that range from space exploration to the authenticity of a Civil War–era land deed. You can “drop in” and contribute to any of these newsgroups . For example, if you are interested in K–12 math curricula, try the k12.ed.math newsgroup. How about pathological behavior? Try the sci.psychology. psychotherapy newsgroup. We will return to them again later for a short demonstration.
· • And finally, there is the world of social media including Facebook and Twitter and these lend themselves to entirely new ways of being used for research purposes. More about these later in this chapter.
More About E-Mail
Imagine it is 1925 and you are sitting at your desk at college, writing a letter to a friend in England. You stamp the letter, mail it, and three weeks later you receive an answer. You are amazed at how fast the mail is and sit down to answer your friend’s new questions about how much you like college and what you will do after you graduate.
Now imagine it is 2012 and you are writing to a friend in England, only this time you use e-mail. From your home, you compose the message, press the send key, and your friend has it almost instantly. Not only does your friend have it, but you copied it to three other members of the research team, including your primary professor. The reply arrives within 20 minutes and “attached” to the message is well-written response to your message and a new paper on the topic of interest.
E-mail works much like conventional mail. You write a message and send it to an address. The big difference is that there is no paper involved. Rather, the messages you send travel from one computer to another in a matter of minutes or hours, rather than in days or weeks, as fast as your voice travels in a telephone conversation.
How should you use e-mail, which is the really big question here? It’s fun for social and family reasons, but it’s an indispensable part of the research process. Imagine having a question about a particular test you want to use in a research study. e-Mail the test’s author. Imagine not being able to find a critical reference. e-Mail the author of that reference (and you should know how to find that author by now given the tips we discussed throughout other parts of this chapter). Imagine not being able to understand a point your professor made in class about a particular statistical technique. With permission, e-mail your professor. This stuff really works.
One note about e-mail. It works because there are servers to which the mail is sent and then distributed. Sometimes these servers break down and mail can be delayed, for an hour or, in some cases when perhaps they have been infected with a virus, for days. Our advice is to have two e-mail addresses, one that you access from school and one of the other many free ones that are available such as those from Yahoo! ( www.yahoo.com ), Hotmail ( www.hotmail.com ), or GMail (from Google). You can always use these as a backup and receive or send mail from there. In many cases, you can even view your other mail account receipts (such as your school mail) within your secondary account.
A huge advantage of Web-based mail is that you can access your mail from any computer in the galaxy. It is always available as long as you have an Internet connection. In addition, as Web-based mailing programs become more sophisticated, they offer features that even fancy commercial mailers such as Outlook might not have, such as being able to (easily) enter a vacation message when you are away from your mail client and want people to automatically be notified. Or, you can send mail through GMail and make it appear as if it is being sent through any other account. Very handy. Many researchers create such new mail accounts for each research or writing project so they can segregate their mail and track it more effectively.
Another note: A host of roadblocks have been introduced along with the millions of e-mails that appear every day in mailboxes around the world in the form of spam, adware, viruses, and other nefarious mechanisms for unscrupulous people to gain access to your privacy. No matter how you do it, take advantage of some of the relatively inexpensive commercial products and install them on your home computer. For the most part, your college or university should be taking care of these concerns at some central location. But for you, it is critical (and almost inexcusable) to have some type of effective and current (and this is really important) way to keep your machine free of viruses and other junk.
And yet another note! In your electronic workings these days, you will see reference to the “cloud.” Cloud-based computing is coming—it’s where data, e-mail, and other information are stored on a remote computer (known as a server), so there is nothing locally available on your desktop. Everything, in other words, is Web based including applications (much like Google Docs is today).
The advantage? Clearly, you can do anything from any connected computer. No more new disk to install when applications change; rather, you would work on a subscription basis and every time a new version of Microsoft Office is released, the changes are right there the next time you open it up. It should be cheaper and more readily available (remember, being connected is everything) and, no more backing up (well, sort of). The cloud system you use stores your data in a safe place.
OK, so what’s the drawback? Although we are told otherwise, oops!—there goes the server and there goes everything you created. While cloud computing enthusiasts speak to the reliability and safety of the system—and it is there—you and I both know that someday it will fail. The lesson? It’s the future, but be sure to use whatever local backup system is available as well.
An Introduction to Usenet (News) Groups
Here’s a topic that interestingly enough many people do not know much about. It’s interesting since newsgroups are such an immense source of information and there are so many from which to select.
Imagine being able to find information on more than 100,000 topics, ranging from stereo systems to jokes (censored and otherwise) to the ethics of law to college football to astronomy. Where would you be able to find a collection of such diverse information that can be easily accessed? You guessed it—the Internet and the various newsgroup sites that ship news each day around the world. The news that fits in one category, such as college football or the ethics of law, forms a newsgroup (also called a group). A newsgroup is simply a collection of information about one topic. Once again, surprisingly, very few students are aware of and use newsgroups.
To help manage the flow of articles, news sites are managed, moderated, administered, and censored by system administrators who work for institutions such as universities and corporations. The newsgroups from which you can select news are those made available by the system administrator and more often than not, the system administrator has to give approval before you are allowed to join and contribute.
What’s in the News?
Newsgroups are named and organized based on a set of rules. The most general of these rules has to do with the name of the group itself. There is a hierarchical structure to a newsgroup name, with the highest level of the hierarchy appearing in the left-most position. For example, the newsgroup name k12.ed.tech means that within k12 (the general name for the kindergarten through twelfth-grade newsgroup), there is a subset named ed (for education) and within that another subset named tech (for technology).
Table 3A.5 is a sample of some newsgroups: what these groups are named, the general area they cover, and examples of what is in each of these groups. Originally, all newsgroups started with the .net suffix. Then, a renaming of newsgroups occurred in 1986 and there were seven main groups; .comp, .news, .sci, .rec, .soc, .talk, and .misc. Humanities (.hum) was added so that the number of primary newsgroups was finalized (for now) at eight. The suffix .alt represents all other newsgroups that do not have a clear place in any other groups (and sometimes jokingly is meant to represent Anarchists, Lunatics, and Terrorists due to the subversive and anything goes nature of .alt newsgroups).
Newsgroups can be small or huge discussions of just about any topic.
Table 3A.5 The Big newsgroups prefixes
| Newsgroup | General Area | Examples |
| Alt | Everything that doesn’t fit anywhere else and certainly lots of stuff out of the ordinary | · • alt.actors.dustin-hoffman (welcome back to the graduate)· • alt.amazon.women (xena, the warrior princess and more)· • alt.anything (guess) |
| Comp | Information about computers, computer science, computer software, and general interest computer topics | · • comp.ai (danger! will robinson!—all about artificial intelligence)· • comp.compression (a discussion of ways to compress or reduce files)· • comp.software engineering (so you want to design a new chip?) |
| Hum | Discussion of issues in the humanities | · • humanities.classics (more about the classic texts)· • humanities.language (discussion about languages and how they fit into the study of the humanities)· • humanities.philosophy (all about the great masters and their ideas) |
| Misc | A catchall of topics and ideas | · • misc.forsale (kind of like a garage sale online)· • misc.books (discussions about books and wri ters)· • misc.invest (how and where to invest your hard-earned money) |
| News | Information about news, newsgroups, and the newsgroup network | · • news.admin.censorships (all about what should and shouldn’t be on the Net)· • news.admin.net-abuse.email (don’t like all that junk e-mail? come here for advice)· • news.accounce.conferences (where to go to be seen) |
| Rec | Information about recreation, hobbies, the performing arts, and fun stuff | · • rec.sport.swimming (make a splash)· • rec.bicycles.racing (what cool stuff to buy for your bike to go faster)· • rec.skydiving (take an extra ‘chute) |
| Sci | Information about science, scientific research and discoveries, engineering, and some social science stuff | · • sci.astro (astronomy)· • sci.cognitive (so that’s what you’re thinking!)· • sci.skeptic (ufos do exist!) |
| Soc | Information about the social sciences | · • soc.couples (people getting along)· • soc.penpals (why people write to one another)· • soc.misc (stuff that doesn’t fit anywhere else) |
| Talk | Discussion of current affairs | · • talk.atheism (about atheism)· • talk.rumor (rumor central)· • talk.radio (find out about Air America, Sean Hannity and more) |
To see how a newsgroup works, let’s follow an example of someone who is interested in educational technology. Almost every browser, such as Firefox, Chrome, or Internet Explorer, comes with its own reader built in and ready to go, but most browsers also come with a groups function that is even easier to use, as you can see in Figure 3A.11 . These tools allow you to read existing news and to post new messages.
The first thing you need to do when you are ready to access a newsgroup is to subscribe to it. Your e-mail program or browser (such as Internet Explorer) can do this, or in some cases you may need a separate news reader . From the list of newsgroups, you can select the ones to which you want to subscribe. Each time you go to the newsgroup, you will get the updated version of those newsgroups, including all the news that has been added to that group since the last time you opened it.
The next step would be to open the k12.ed.tech newsgroup and examine the contents, as shown in Figure 3A.12 (we used Google as our reader). Within newsgroups, you will see a listing of topics open for discussion, each one started by an individual as a source for more information, a place to meet electronically, discuss issues, and so forth.
If someone wants to participate in a certain newsgroup, he or she can add a new topic at this level, or go into an existing newsgroup and make a contribution.
Figure 3A.11 The Opening screen for Google groups where you can search for groups, start one of your own or explore the most popular ones.
Google™ is a registered trademark of Google Inc.
Figure 3A.12 The newsgroup is a wide-open community where everyone is welcome to contribute and learn.
Google™ is a registered trademark of Google Inc.
Using Mailing Lists or ListServs
Another really neat way to use the Internet is a great source of information. You can sign up (subscribe) for a listserv discussion group, which is an automatic depository for information. If you subscribe, you receive everything that the list receives. A listserv is also known as a mailing list.
For example, if you belong to the K–12 educational technology mailing list, then each time someone sends mail to that list, you will receive it as well. There are more listservs than you can imagine, and it will take some exploration to find out which ones best fit your needs.
To subscribe to a mailing list, you need to send a message to the list’s administrator. As soon as you do that, a constant stream of messages will come your way. Be careful—if a list is very active, you can receive hundreds of messages in any one day. If you go even a day without checking your mail, your electronic mailbox is likely to get so full of messages that you won’t be able to read anything! Imagine your real mailbox outside your apartment or home. When it gets stuffed full, it is very difficult to pull out any one piece because the mail is packed so tightly. You would need a bigger box (more storage space), or you need to empty the box before it gets so full. Such is the case with an Internet mailing list: Either get a larger e-mail box (ask for more storage space from the system administrator) or check your mail more than once a day.
At Catalist ( http://www.lsoft.com/catalist.html ) you can find a guide to the always update list of over 500,000 lists(!), all available to you and me, and you can search by the number of subscribers, the country of origin, and, of course, the topic. Want to spend unending hours at your computer learning about everything from black holes to death rays, this is the place to start.
And, Just a Bit about Web Sites
In the last edition of Exploring Research, this section was filled with information about home pages or Web pages and what is contained on them that might be of use. There was a screen shot of the Library of Congress Web site’s opening page (which is a portal to the entire site at http://www.loc.gov ), but times have changed, as does the information we include here.
Web sites are of course portals to more information but should a researcher use them? The most obvious way is by seeking out information contained at various sites, perhaps including links to other sites and on and on. But, as an active researcher there’s another entirely different function that Web sites can perform.
You can easily, and at little expense, create a Web site of your own, for your own research project. Unlike the early days of Web development and design, today you can create a home for your content easily, quickly, and relatively inexpensively. No need to hire the 10-year-old from down the block or some fancy firm that charges $25 per hour. And, no need to know programming languages such as HTML (Hypertext Markup Language). Today, companies such as GoDaddy (www.godaddy.com ) and Homestead ( www.homestead.com ) are not free, but are often ridiculously cheap (such as $4.99 per month for Homestead). And best of all, these companies and many others provide a wide variety of tools to help you drag-and-drop, modify, spruce up, and generally make your Web site quite professional looking.
What do you do once you have created it? As the home for your research activities, you can conclude such information as
· • The title of the project
· • How to get in touch with the project’s coordinator (probably you!)
· • A history of what is being done, where and how
· • Forms that may need to be completed
· • Data that may have been collected that can be shared
· • Links to similar studies, and
· • Any information that educates the viewer and may encourage him or her to participate.
Using Social Media and Blogs
There’s no end to the imagination of entrepreneurs when it comes to the use of technology to have an impact on our lives, and correspondingly, there is no end to the imagination of researchers to use that technology in their research as well.
What About Facebook and Twitter?
You know about Facebook and Twitter, but could you imagine using these tools in a research setting? There are many others, but we’ll look at just these two. Here’s how to use them.
Facebook is a social networking tool that allows users to form groups, communicate with each other, and even play games. With over 600,000,000 active users, it is an amazing way to get like-minded people together to discuss and participate in research where some common interest is maintained. You would be well suited to begin a Facebook group based on your own research interests and reach out for others who have interests that are similar to your’s.
And, of course, Facebook participants can very well become participants in a study as well. Facebook is a magnificent naturally occurring laboratory to study (mostly) young people’s ideas and actions as they exist in virtual and real-time groups. Researchers from Harvard, Indiana, and Carnegie-Melon are all using online subject samples to collect data and test hypothesis.
Twitter is another social networking tool that allows users to create 140-character messages and then allows those messages to be sent out to anyone who is following the author. Sometimes small potatoes and only 10-20 followers. But sometimes, followers number in the hundreds of thousands.
Among other ideas, you can of course follow a researcher in whose work you are interested by simply signing up to follow him on Twitter (you need a Twitter account to do any of these things). Then, each time he or she creates one of those 140 character messages, it comes to you and the hundreds or thousands of other people who signed up on his or her list.
Another way to use Twitter is to find out what is being written as people are being followed by searching on this huge and vast electronic archives that are available. For example, if you wanted to know what people were saying (or Tweeting) about nursing education, you can use Twitter’s simple search box on the main page and enter the words “nursing education” (using quotes since you would want the search to return for both terms together, not each one separately). Or, if you want to dig even deeper, go to the advanced search form (look for it under Help or at http://search.twitter.com/advanced ) as you see in Figure 3A.13 . And, it’s simple enough to find people—just click the Find People button on the main page.
Writing the Literature Review
It is now time to take all the information you have collected using all the tools you have learned about in this chapter and somehow organize it so it begins to make sense. This is your review of literature, and now you actually need to write it (horrors!). Here are some writing hints.
First, read other literature reviews. There is no arguing with success. Ask a student who has already been through this course or your adviser for a successful proposal. Look carefully at the format as well as the content of the literature review. Also, look at some of the sources mentioned earlier in this chapter, especially sources that are reviews of the literature, journal articles, and other review papers.
Figure 3A.13 Using Twitter help.
Second, create a unified theme, or a line of thought, throughout the review. Your review of literature is not supposed to be a novel, but most good literature reviews build from a very general argument to a more specific one and set the stage for the purpose of the research. You should bring the reader “into the fold” and create some interest in where you will be going with this research that other people have not gone.
Third, use a system to organize your materials. Most reviews of the literature will be organized chronologically within topics. For example, if you are studying gender differences in anxiety and verbal ability among adults, you would organize all the references by topic area (anxiety and verbal ability), and then within each of these topics, begin your review with the earliest dated reference. In this way you move from the earliest to the latest and provide some historical perspective.
Fourth, work from an outline even if you are an accomplished and skilled writer. It is a good idea to use this tool to help organize the main thought in your proposal before you begin the actual writing process.
Fifth, build bridges between the different areas you review. For example, if you are conducting a cross-cultural study comparing the ways in which East Indian and American parents discipline their children, you might not find a great deal of literature on that specific topic. But there is certainly voluminous literature on child rearing in America and in India and tons of references on discipline. Part of the creative effort in writing a proposal is being able to show where these two come together in an interesting and potentially fruitful way.
Sixth, practice may not always make perfect but it certainly doesn’t hurt. For some reason, most people believe that a person is born with or without a talent for writing. Any successful writer would admit that to be a class-A basketball player or an accomplished violinist, one has to practice. Should it be any different for a writer? Should you have any doubts about this question, ask a serious writer how many hours a day or week he or she practices that craft. More often than not, you will see it is the equivalent of the ballplayer or the musician. In fact, a writer friend of mine gives this advice to people who want to write but don’t have a good idea about the level of involvement it requires: “Just sit down at your typewriter or word processor, and open a vein.” That is how easy it is.
So the last (but really the first) hint is to practice your writing. As you work at it and find out where you need to improve (get feedback from other students and professors), you will indeed see a change for the better.
Summary
There’s a lot to know about this selecting a problem topic and doing the necessary background research and it just begins when you have some familiarity with your field and some experience using both online and offline resources. Finding a topic and a question that works for you (in every sense of the word) is a real challenge and often an obstacle for beginning students and beginning scientists. Take your time, talk to your colleagues and your faculty, and make it into an exploration looking for the gold that represents a topic that will carry you to a new level of intellectual growth.
Exercises
1.
Make a list of 10 research topics that you would find interesting to pursue. These can be any topics dealing with education or psychology which you might glean from newspapers, radio and television news, magazines, research journals, and even overheard conversations. Rank these various ideas by level of interest, and for each of the top five write one sentence explaining why it appeals to you.
2.
Take the idea that you ranked no.1 in exercise 1 and do the following:
· (a)Write a one-paragraph description of a study that incorporates that idea.
· (b)List the steps you could take in reviewing the specific literature relevant to this topic.
· (c)From this idea, generate three more questions derived from the original question or idea.
3.
Use the idea that you ranked no. 2 in exercise 1 and do the following:
· (a)Locate a related reference from a journal and write out the complete citation.
· (b)Locate an abstract from a study that focuses on the topic.
4.
Find ten other sources of information about any of the topics you ranked in exercise 1 and write out the complete citation for each. Try to complete a set of other sources that is as diverse as possible.
5.
Go to your library and find five journals in your field of study. After you have located the journals, examine them to determine:
· (a)What type of articles are published (reviews of literature, empirical studies, etc.).
· (b)Whether the journal is published by a professional organization (such as the American Psychological Association) or by a private group (such as Sage Press).
· (c)The number of articles in each journal and if there is any similarity in the topic areas covered within each issue of the journal.
· (d)How often the journal is published and other information about its editorial policies (e.g., guidelines, features).
6.
Select any topic that you are interested in and use three different search engines to obtain on-line information. How do the results differ? Which one gave you the most interesting and useful information? How might you revise your search terms to get the same degree of usefulness from other search engines?
7.
Visit Google Groups at http://groups.google.com . Type in a topic of interest for you next to the “Search for a group” link and click on the link. Write down the title of the group or the group e-mail address.
8.
Find three abstracts from recent research journals. For each abstract identify the following:
· (a)The purpose
· (b)The hypothesis
· (c)The type of study (e.g., correlational, experimental)
· (d)The conclusion
9.
You have been assigned the topic of gender differences in adolescent development for a research study. Formulate five research questions that address this topic.
10 .
What are some helpful things you can do to help you figure out if your first idea for a research study is the best one?
11.
Use the Internet to find five references on any of the topics in which you have an interest (as you defined in earlier questions).
12 .
What purpose do the following search commands serve?
· (a)phonebook:
· (b)*
· (c)“blended families families” (entering one word twice)
13 .
What are some potential advantages to reading peer-reviewed journal articles instead of relying on information obtained through other online sources such as Wikipedia? What are some potential advantages to using Wikipedia?
14 .
Indicate which of the following are general, which are primary, and which are secondary sources:
· (a)Encyclopedia of Psychology
· (b)Time Magazine
· (c)Statistics for People Who (Think They) Hate Statistics (textbook)
· (d)Journal of Sport Health
· (e)Facts on File
· (f)A review of Freud’s dream interpretations
· (g)Dissertation Abstracts Online
15 .
A recent study found that college freshmen who participated in an occupational engagement intervention wherein they e-mailed professors, learned ways to increase their information about careers, and wrote about their experiences of learning more about careers scored significantly higher on a measure of career decision-making efficacy than students in a control group. How might you work from or replicate this study without being redundant?
16 .
Look up the keyword phrase “test anxiety” in Google Scholar. What are the first three titles that appear?
17 .
Wang and Amato found in their 2000 study that divorce adjustment was significantly related to income levels, age, remarriage status, and previous attitude toward marital dissolution, among other variables. What are three more variables you could examine in relation to divorce adjustment?
18 .
Why is it helpful to review literature before finalizing your hypothesis?
Online. . .
The Gale Directory of Online, Portable, and Internet Databases
The Gale Database Directory at http://library.dialog.com/bluesheets/html/bl0230.html provides detailed information on publicly available databases and database products that are accessible through an on-line vendor or the Internet.
The GPO Database List
Want to see how a huge amount of data can be organized and made easily accessible to the online user? Check the U.S. Government Printing Office (GPO) database online at http://www.gpoaccess.gov/databases.html . You can find out about everything referred to in a specific House or Senate session through the Congressional Quarterly or what bills have been passed. Best of all, this whole collection illustrates what power is possible when the Web and databases come together.
The National Library of Medicine Databases
The National Library of Medicine provides a wide variety of past and present resources related to the biomedical and health sciences at http://www.nlm.nih.gov/databases/index.html . The format of databases varies, including being searchable to just bibliographic citations to full text. You’ll find tons of stuff for the social and behavioral sciences researcher as well as the aspiring nuclear scientist.
Choosing the Best Search Engine for your Information Need
Noodle Tools compares different search engines and recommends the best ones based on your need (identifying a topic, narrowing a topic, finding primary sources, by topic, historical or current, etc.) at http://www.noodletools.com/debbie/literacies/information/5locate/adviceengine.html .
Ready ‘Net Go!
On this Web site, available at http://www.tulane.edu/~lmiller/ArchivesResources.html , Tulane University exhibits a “meta index” with links to nearly every archival resource available. The site contains general search tools as well.
