By Dennis D. McDonald

On May 3, 2014 the Washington Post published Unrequired Reading, an exposé of what really happens to the 4,291 reports that are required by law to be regularly submitted to the U.S. Congress.

What really happens to the reports?  Not much, according to the Post.  Sometimes the reports contain valuable information and are used as intended as an aid to congressional oversight. Many times reports are submitted by law by program staff to Congress and never read or examined, the many hours of personnel resources devoted to their production being wasted. Still others are never even written and submitted to Congress—and no one notices.

(Pause for knee-jerk Washington-bashing.)

Understanding what’s going on here requires consideration of several factors:

1. Things change.
2. Reports don’t exist in a vacuum.
3. It’s hard to predict in advance who will find what information is valuable.
4. If you have to wait to see a report to know if a program is performing it’s already too late.

One.  Things change.

One frustration described by the Post is that some reports are still being generated even when the need has passed. For example, reporting on the Association of Spanish War Veterans continued long after the last veteran of that war had died.

What is needed is a way to “sunset” the report generation process but even this is resisted. Based on the discouraging experience of the Obama administration trying to reduce the number of reports it’s not unusual for political roadblocks to be created regarding deletion. 

What we don’t know from the Post article is why anyone would defend useless reports.  Are they really useless?  Does someone still find them useful?  Have they turned into consultants’ and staff employment programs?  Or would the programs being reported on feel “slighted” if a congressionally mandated report were no longer required?

We can’t really tell which of these might be the case. Uncertainty around the answer to question “useful to whom?” appears to be key.  If the program itself that generates the report is no longer useful, what does that say about the report?  Is it more important to go after the program or the report?  If the program is useful but the report about the program isn’t, what does that say about the reporting process and why it exists in the first place?

It’s unlikely that anyone will seriously suggest that Congress should not have oversight responsibilities.  The appropriate question is, what role should reporting by the program agency play in the oversight process?  Can the program evaluate itself, or should the program be reporting objective performance measures that support an effective evaluation?

These are not trivial questions as student of government performance measures will attest.  The programs and the environments in which these programs operate will continue to change. Reporting processes will have to change as well.

Two.  Reports don’t exist in a vacuum.

In simpler times it might have been reasonable to think that an annual “report” could possibly encapsulate the performance of a program in a single document.  As the size, number, and complexity of government programs have expanded, the size, number, and complexity of reporting has expanded as well.

According to the Post article the number of reports has expanded and increasingly these reports are delivered electronically.  But the reports are still designed for human consumption.  It’s not unusual to find such reports in .pdf form, configured to resemble the structure in appearance of a print a document even if most of the reports will actually be “consumed” via a PC or tablet computer.

As suggested above, though, such reports do not exist in a vacuum.  Legislators need not wait till an annual or biennial report is issued to assess the performance of the programs they oversee.  Information about program performance can be obtained from a variety of sources internal and external to the agency running the program.  Constituents and lobbyists can be contacted.  Publications and social media can be monitored.  Watchdog and advocacy groups can be polled. By the time a report is formally submitted the information it contains might already have grown stale, superseded by more current and varied sources.  In this scenario, especially if it becomes increasingly possible to monitor government programs performance from sources other than a limited annual report, what’s the point of requiring the report in the first place?

Three.  It’s hard to predict in advance who will find what information is valuable.

One the first things I learned when researching how professionals use information systems is that you have to allow in system designed for serendipity.  No matter what you think you know about what information people need to do their jobs or make decisions, no matter how you rank that knowledge in system design, users are going to get their information from wherever they please.  Not only that but it’s also hard to predict what data you put into a report will be most useful.  In fact, that is one of the main motivators behind the “open data” movement.  Yes, you want to generate and manage data in connection a program in a way that explicitly supports the program. You also want to make that data available to users so that new and innovative uses of the data can be developed.

The data that goes into a report to Congress might also have uses above and beyond the potentially narrow goals of the report that contains the data.  If that is the case, we need to be sure the report containing the data is widely available; that’s one goal of legislation introduced by Rep. Mike Quigley (D-Ill.) to publish congressional reports on a single web site.

Four.  If you have to wait to see a report to know if a program is performing it’s already too late.

Any government program worth its salt should already be providing data in an open and transparent fashion that allows any citizen — not just congressional oversight committees — to evaluate the program’s performance.  It should be doing this in a regular basis on the program’s web site. The data should be provided in a standard fashion along with tools that support downloading, re-use, sharing, and searching.

You shouldn’t have to wait for an after-the-fact report to be able to judge a program’s effectiveness. Still, you may not want to give up this requirement for special reporting even if you do provide ongoing data that are designed to track defined performance measures.  Why not?  There is great value in bringing together a group of experts on a regular basis to intensively and objectively review the program and ask the tough questions:

• Is the program being run effectively?
• Is the program doing what it set out to do?
• Should the program be extended or cut back?

These are questions that should be addressed through concentrated and objective thinking by experts in a context that is separate and independent from the day to day running of the agency’s program.  In other words, don’t throw out the requirements for report; make the reporting process meaningful and useful.

Summary and Conclusions

No doubt much of what is reported in the Post article is true and continues to be true. The effort involved in generating many government program reports is wasted. Reports that are generated may never be read. Efforts to do away with unnecessary programs and reports are met by resistance from interest groups inside and outside the government. Still, I wouldn’t throw the baby out with the bathwater. Communicating what a program does and how well it does it is an important function. Since the means available for doing this are changing, we need to adapt to the new realities around how performance data are gathered, processed, and shared. We need to rethink how reports to Congress are done and act accordingly:

1. Terminate useless reports.
2. Make reports accessible to all, not just a limited congressional audience.
3. Make data associated with reports available in a standard and reusable machine readable form.
4. Generate reports in an objective and professional manner.
5. Follow up on how reports are used and use this information to make them more useful.

Copyright © 2014 by Dennis D. McDonald