A Candid Examination
of the Performance of
Science in Government
Scientists and statisticians discuss integrity of science
in government agencies and the contracts that
underlie federal policies
by Roberta Spalter-Roth and Janene Scelza,
ASA Research and Development Department
Scientific expertise, including the collection, analysis, and dissemination of data, can and should play a critical role in informing federal government policy. In a decision-making arena often dominated by partisans and interest groups, however, there is an ever-present potential for the omission, suppression, or abuse of scientific data and analysis.
Given this potential for breaches in scientific integrity, ASA convened a meeting of sociologists employed in federal government and nonprofit research settings. The May 14 event at the ASA office in Washington, DC, was attended by more than 25 sociologists from a wide range of government agencies and nonprofit research organizations who confidentially discussed their experiences with government and contract research.
The meeting was part of an ASA Council initiative recommended by Past President Frances Fox Piven, who, like many sociologists, was concerned about the possible misuses of social science measures, data, and expertise by policymakers in the federal government. Many of these concerns had been spawned by press reports on various incidents and congressional hearings on topics ranging from NASA and global warming research to the National Institutes of Health and cancer research.
In August 2007, Council established a Subcommittee on the Production and Uses of Federal Social Science Data that included Council members and Executive Office staff. Council requested that Executive Office staff gather relevant information from ASA members and sociologists conducting research within or on behalf of government agencies or who perform other government-funded research. Council also recommended ASA’s participation in the Union of Concerned Scientists (UCS), an independent organization that monitors allegations of violations of scientific integrity in the federal government but which primarily has focused on research in the natural sciences.
Challenges and Difficulties
Attendees at the May meeting spoke candidly about patterns of management that might either lead to abuses of scientific integrity or the suppression of research with the potential to inform government decision-making. A major discussion theme was how executive agency policy is often made without the benefit of research, even when such research is requested by the agency leadership. According to the meeting’s attendees, multiple layers of federal review at mission agencies and the Office of Management and Budget (OMB), as well as long delays in study approval, frequently make the conduct of relevant research too difficult, too expensive, or untimely.
Attendees cited difficulties related to government processes, the qualifications of some OMB staff, and the political motivations of policymakers. Many attendees expressed considerable concern that governmental processes inhibited the contributions of research to policy, whether or not those processes were or appeared to be deliberately manipulated by policymakers to achieve a desired outcome.
Several participants expressed concern, for example, regarding the qualifications of some OMB staff to review the proposed social science methodology or to determine reasonable standards of study design. Sociologists reported instances where OMB staff members insisted that the design meet standards beyond the scientific norms of the field (e.g., requiring unobtainable response rates on surveys).
The challenges of timeliness were also a contributing roadblock cited by the meeting’s attendees. Policy-related research is often "held hostage" by the length of the mission agency review process. Policy decisions are often made before research can be made available.
Sociologists also reported that agency researchers are often not involved at the inception of programs to develop designs for evaluation. Instead, researchers are frequently called in too late in the process to effectively contribute to program design or evaluation. In cases where timing is not an issue, funding may be a challenge. Researchers are often told that there is insufficient funding within the agency for an appropriate evaluation to be conducted even though it is often critical to sound policy formation.
A Seat at the Table
Aside from timing, funding, and other challenges, participants lamented over the government’s frequent failure to give researchers a seat at the table to inform federal policy. Instead, sociologists cited the tendency for political perspectives to predetermine policy outcomes, with or without research input. Attendees reported being told explicitly that research would not be conducted, even when the program or policy in question had generated researchable questions or concerns.
Even when federal research does produce findings, researchers reported that their reports are sometimes edited to support the directions of current federal policies or programs. Alternatively, research is left to gather dust on the shelf.
Independent publication of such findings is exceedingly difficult for researchers in government agencies and risky for the researcher involved. Similar concerns were voiced by a recent UCS report on violations of scientific integrity at the Environmental Protection Agency (EPA) based on survey responses of 1,586 scientists (see www.ucsusa.org/scientific_integrity/).
Many of those participating in the ASA-convened discussion had many years of federal experience as research sociologists. The issues they raised were endemic to the federal government, more acute under some administrations than others, but nonetheless, they reported, were ever-present.
This suggests that disciplinary societies such as the ASA and watchdog organizations such as the UCS have potential contributions to make on two fronts: (1) considering how to improve research management and policy development within the federal structure to help strengthen the potential for social science research to contribute to effective decision making so that agencies are responsible stewards of U.S. taxpayer money; and (2) challenging specific instances of egregious violations of scientific integrity within that structure when they arise. The participants at the May 14 discussion were enthusiastic about a series of future meetings to explore how ASA might take the next steps in this long-term effort.