READING

Panel Comments: Patrick Clark

Good evening, Thank you for having me. I’m Pat Clark and I'm on the evaluation policy staff at the National Institute of Justice. It’s a pleasure to be a part of this effort. With these first slides, let me take you through how evaluation policy might be developed in the Department of Justice (points to slide with organizational chart of the Department of Justice).

Some say that this is the largest law firm in the world, and over there on the left hand side, you’ll see the Civil Division under the Associate Attorney General, who’s currently acting, under the Deputy Attorney General, who’s currently acting…and the Attorney General who may be appointed any day now. The Office of Justice Programs is where the National Institute of Justice is situated and, as you can see, under the Office of Justice Programs, the National Institute of Justice is embedded in the programs and bureaus of OJP. We consider ourselves to be the research, development and evaluation arm of the Department of Justice, but it’s a very small arm in that regard. We are to consult, and often do assist the program offices in evaluation work, but that’s often something that we wait for.

Here is an organizational chart of NIJ. I’m situated in the Office of the Director, and I’m now what’s referred to as an Evaluation Specialist. On the right hand side of this chart, you’ll see the Office of Science and Technology (OST) in the National Institute of Justice. They are primarily in the business of forensics and technology development, and follow a scientific R&D kind of approach.

Over the last few months, since I’ve been moved from the Office of Research and Evaluation to the Office of the Director, we’ve engaged OST in a process of looking at evaluation in more in-depth terms than what they’re used to. In R&D and technology development, evaluation is often a matter of usability and human factors, and what I call ”Dick Clark” evaluation. That is usually the extent of their evaluation research – it’s easy to dance to, it’s got a good beat. Well, we’re now in the process of discovering with OST what outcomes might be evaluated.

Policy in the National Institute of Justice is primarily set by the Director, who is a appointed by the President and confirmed by the Senate. Dr. David Hagy, who you saw at the top of the previous chart, has been nominated, but has yet to be confirmed. But, a large portion of the policy regarding evaluation that we have in place, in use, I think is a function of outside pressure. And, our good friends at GAO are responsible for some of that pressure. In recent years, they’ve issued three reports critical of evaluation research at the National Institute of Justice, as you see by these titles (on the slide) here. I’m sure you can pick up these reports at your local bookstore, but in no uncertain terms, they sent a message to NIJ to improve our evaluation research.

My position was created as a result of these reports. So, I’m actually quite thankful to GAO for that nudge - To assess evaluation processes, develop approaches, and develop appropriate strategies. And, I’m all about that these days.

Following that in 2005, the National Academies, Committee on Law and Justice formed a committee on improving evaluations of anti-crime programs, primarily looking at National Institute of Justice work. We’re were nudged by them with 23 recommendations to improve our work in evaluation. That’s having an impact on the way we do evaluation research. 

Also, in the spirit of full disclosure, there is this thing called, PART. It brings a tear to my eye to tell you that the NIJ is rated by OMB as only “adequate.”

What types of evaluation- related input would NIJ welcome? I’ve been thinking about this since receiving the invitation to be part of this forum. I think AEA is doing a fine job in that regard. Many of our staff attend your conference on an annual basis. In recent years, there have actually been some resources for staff to get in-service training and they attend the workshops at AEA. I was left with putting your charter up for this slide to say, in reaffirming your charter, you are doing a fine job. But, if you want to break into new areas, I may be able to give you some ideas.

We spend a lot of our time at the Institute for Justice trying to evaluate programs that are enacted through the process that was laid out here. And, in the area of crime and justice programs, those enactments usually come through crime legislation. You may have heard of the last large one, the 1994 Crime Act of the Clinton Administration. That actually created a number of offices, including the Office of Community Oriented Policing Services and the Office of Violence Against Women. The Bureau of Justice Assistance (BJA) was probably the first crime program office and was created under the 1968 Omnibus Crime Control Act. They’re primarily responsible for what’s referred to as “Byrne grants.” These are block grants to local governments and police departments. They spend these funds across about 30 purpose areas.

That gets me to the challenges that we face - and I’m whining here - as it relates to evaluating criminal justice policies at the Federal level. Federal policies, at least in criminal justice, are often enacted, funded and implemented without any regard to evaluation. It’s a political process that results in a crime act. Crime act formula funds are distributed through agencies like the Office of Victims for Crime, and the Office of Violence Against Women with an array of purpose areas that are very difficult to operationally define and evaluate.

Block grants are similarly awarded across many objectives without any provision for evaluation. And federal programs, especially in the area of crime and criminal justice, overlap in jurisdiction and in services. In one location there will be a little bit of money applied toward violence against women, maybe to hire a prosecutor. You’ll have a victim’s compensation board. You’ll have a community oriented policing service. And, it’s difficult to assess the effect of any of them in terms of outcomes.

Specific program funds are marginal, minimal at best. A prosecutor is a good example. For the Violence Against Women-STOP program, a local jurisdiction can provide a prosecutor for about $85,000. But $85,000 a year may not make a lot of difference in terms of outcomes. Funds for the purposes of evaluation are often limited and seldom sufficient. Our base for discretionary spending on research grants will probably be about $10 million this year. And unfunded mandates are very popular and could unexpectedly use some of that discretionary base.

Lastly, earmarks are crossed out on this slide. But, I’ve heard they’re going to come back with names of sponsors associated with them. Earmarks exist with few conditions and no requirements for evaluation. This all makes it very difficult to do evaluation work. Do you feel my pain?

What are the means to which AEA could provide some input? A large portion of what you do is having an impact, especially with young staff who are seeking education in the areas of evaluation. But as alluded to here, we need to help folks out there discover what evaluation is and what it isn’t.

Providing objective and independent information in the system is, I think, a laudable effort. And, AEA can function very well in that regard, and educate us on contemporary and cutting-edge and innovative methods. It’s not uncommon for NIJ to publish a solicitation, and get a hundred applications in response. And, half of those might propose what Tom Cook calls the “workhorse” (non-equivalent group quasi-experimental) design. We just finished - reported out here yesterday - our first quasi-experiment with propensity score matching - so you are making inroads.

The Policy Task Force is really a great idea, I wanted to reiterate, your efforts are laudable.

Timing of conferences has been a topic of discussion. I’ve been back and forth from another conference in town this week. And, we have to work on timing of important conferences. I’m leaving town tomorrow to head to the American Society of Criminology. If we could all collaborate on schedules, I think we’d have a heck of an impact. For instance, APPAM. One of the evaluators that I’ve met through my association with Stephanie Shipman and Federal Evaluators Group is at the APPAM conference this week. Many of the folks in your target group are at that conference right now, and many of them being trained in economics and in public policy - but they really need your help. NAPHA is meeting this week as well, and their members won’t have the opportunity to attend an AEA workshop.

The next slide is an e-mail that I received last Friday. I’m inundated with these constantly. These are offers to come out and be trained with in-service opportunities. I think that AEA could get into the business of doing this kind of one-day forum. Timing is everything, but across the span of a year, staff often have an opportunity to get out of the office for a few days and concentrate on a particular topic. As an example, I just had the opportunity to go to Alexandria, Virginia and attend a workshop on concept mapping. That wouldn’t have happened if it hadn’t been available in town.

And then there's this place on the Internet – www.expectmore.gov. If you want customers, there’s a list of agencies and organizations on that website right now that have been assessed as “inadequate,” or who haven’t performed, with “results not demonstrated” and they need your help.

How does the presidential election affect... There’s an opportunity coming here. The transition is beginning, or has begun. Political people are leaving government daily. Political appointees are leaving and in their place will be career people for some duration of time while the new Administration is moving into place. Career people appreciate what you’re talking about and I think you may be able to take advantage of this opportunity.

You could also create new opportunities to interject your message. In another life I was a lobbyist for a citizen’s based state-wide charter. Even though you may not have money to buy a ticket to the party, you may still meet with executive and congressional staff. Create teaching opportunities in these kinds of forums and meet with career agency staff in the interim. Go to and participate in forums, and submit some testimony, even though you may not be able to read it aloud on CSPAN, it gets on the record. And, encourage your membership to become active at the local level. Interject yourself in the process of program development, especially at the local level.

What factors may likely facilitate or hinder? How about an administration that is looking for input? How about an administration that is not? I really can’t predict which way we’re going to go with that. But I wish you all the luck in the world. But, please avoid being used and exploited as it relates to politics. Affirm your charter at every turn, and I think that will serve you well. Thank you very much.

Panel comments: Bernice Anderson