Proprietary PCAOB research, audit quality and ping-ponging deficiency rates
Some great new research and a fabulous conference in Chicago made me think about how research influences our perceptions about whether audit quality is improving and how.
If you can keep your head when all about you
Are losing theirs and blaming it on you,
If you can trust yourself when all men doubt you,
But make allowance for their doubting too;
If you can wait and not be tired by waiting,
Or being lied about, don’t deal in lies,
Or being hated, don’t give way to hating,
And yet don’t look too good, nor talk too wise:
If you can dream—and not make dreams your master;
If you can think—and not make thoughts your aim;
If you can meet with Triumph and Disaster
And treat those two impostors just the same;
If you can bear to hear the truth you’ve spoken
Twisted by knaves to make a trap for fools,
Or watch the things you gave your life to, broken,
And stoop and build ’em up with worn-out tools…
If— By Rudyard Kipling
I am back from Chicago and making a quick turnaround to travel next week to the UK to cover the Cambridge Disinformation Summit in person. I can’t wait to see my Cambridge colleagues again. Frequent readers of The Dig may recall that I was a guest lecturer last September, in residence for a week in Cambridge, co-teaching the audit module for its Executive Masters in Accounting Program with Professor Michael Willis.
I had to come back to Philadelphia this week so my dog Bubbles could get back to familiar surroundings and stop being encouraged to eat mom's leftovers. I also had to complete my taxes after helping mom and another family member with theirs.
No rest for the accountants in the family!
I chuckled with Mark Maurer's mention on LinkedIn of the sequel to "The Accountant":
I was shocked, not, when preparing my own tax returns to see how much I spent last year on my drug of choice, coffee. It was more than I donated to charity!
The Dig runs on Dunkin', La Colombe, Starbucks, and Bad Ass!
In 2025 I plan to spend more on charitable contributions and less on coffee, given the great need in so many places. My charitable focus is on journalism, homeless and migrant support, and immigration rights. Some of my favorites are Mississippi Free Press, SOME, Central Union Mission, Al Otro Lado, Casa Mariposa, No More Deaths, and Annunciation House.
On April 10-11, I attended the "2025 Antitrust and Competition Conference - Economic Concentration and the Marketplace of Ideas," at the Stigler Center at the University of Chicago Booth School of Business. There were many interesting panels, but today I will mention one of them in the context of academic research on accounting/audit issues and conflicts of interest.
I am going to dig into this panel from Friday and a new working paper, "The Conflict-of-Interest (CoI) Discount in the Marketplace of Ideas," out at the National Bureau of Economic Research from authors John M. Barrios, Filippo Lancieri, Joshua Levy, Shashank Singh, Tommaso Valletti & Luigi Zingales.
You may notice the disclosure statements at the beginning for each speaker.
Professor Luigi Zingales, the Director of the Stigler Center, sent out a reminder of the Center’s disclosure policy to all participants and in-person attendees:
…we wanted to reiterate the Stigler Center’s disclosure policy ahead of its annual antitrust and competition conference. The idea is to accommodate the antitrust community’s increasing commitment to stronger disclosures of potential conflicts of interest.
Conversations about how potential conflicts of interest can influence scholarship and policymaking have become more frequent among the antitrust community, particularly over the last year. In September, former Assistant Attorney General for the Division of Antitrust Jonathan Kanter discussed what he saw to be the harmful effects of an increasing breakdown between academia and advocacy. Aviv Nevo, the former director of the Federal Trade Commission Bureau of Economics, made similar remarks in November.
Recent scholarship has further clarified the mechanisms by which conflicts of interest can influence academia. Filippo and I have a new NBER working paper discussing how conflicts of interest impact the trustworthiness of economics research. Another paper by Ioannis Lianos focused specifically on the hidden costs of corporate funding of antitrust scholarship and other interactions between academia and industry. In March, the Stigler Center, together with other academic institutions, co-organized a conference in Brussels to discuss how corporate relationships impact antitrust scholarship. The OECD competition committee will host a roundtable on the topic in June.
Concurrent with Nevo’s remarks last November, the FTC Bureau of Economics amended its disclosure policy to ask all speakers participating in its events to provide their disclosures of potential conflicts of interest upon delivering their initial remarks. The FTC policy builds upon the disclosures required by the American Economics Association (AEA), and they are aligned with the ethical guidelines published by Academic Society for Competition Law (ASCOLA).
The Stigler Center’s disclosure policy likewise observes the requirements of the AEA. Following the FTC, the Stigler Center will also encourage all speakers participating in its Antitrust and Competition conference, as well as at all future Stigler Center events, to briefly disclose any potential conflicts of interest before making their first statement. These statements do not have to be long, but they are important.
In particular, following the FTC’s lead, conference participants are encouraged disclose:
Any significant support the speaker received over the past three years from any "interested party." Significant support includes financial support (USD 10k or more) and privileged access to data. As defined by the FTC, an "interested party is any individual, group, or organization that has a financial, ideological, or political stake in the topic(s) related to [the speaker] participation in the event. This should include any support received both directly and indirectly, including funding for centers and institutes that are material benefit to the author(s).”
Any paid or unpaid positions as an officer, director, or board member of any organization whose policy positions, goals, or financial interests relate to the topics associated with their participation in the event.
The disclosure should be specific and, to the extent possible, cite the specific parties that provided significant support. For example, if the speaker worked for clients with a stake in the matter, the speaker should name the clients. If the speaker's center or organization received significant funding from an interested party, the party should be named. If the speaker cannot name the client/party because of a confidentiality agreement, the speaker should state the nature of the support, mention the confidentiality agreement, and then provide as much public information as possible.
If a third party had the right to review the speaker's remarks.
If the speaker's spouse or partner has any such conflicts.
If the speaker has no conflicts of interest, the speaker should say that they have no conflicts to disclose.
The Center disclosed that the Antitrust conference is directly funded by the Stigler Center, which is itself predominantly funded by an endowment from the University of Chicago Booth School of Business. The Center also received a grant of USD 10k from the Knight Foundation to support the Conference.
On the panel Barrios, et al (2025), co-author Tommaso Valletti spoke about their paper. With regard to the use of proprietary data in academic research, the Barrios, et al (2025), paper provides this example:
Companies and public institutions, like the Federal Reserve, offer selective access to proprietary databases, often with the goal of promoting narratives that align with their interests (Berg and Johnston, 2019) while restricting access to data that might support conflicting perspectives (Horan, 2019;Wagner, 2023; Zingales, 2019).
A whistleblower leak of Uber’s internal communications offers a telling example: after completing a commissioned study, an academic requested access to Uber’s data for an independent, unpaid study. In internal discussions, Uber executives expressed concerns that granting access would cause them to “lose editorial control,” but a senior staff member reassured them, saying, “We see low risk here because we can work with [...] on framing the study and we also decide what data we share with him” (Lawrence, 2022).
During the panel Professor Stephen Haber and law professor Bill Kovacic talk a lot about how universities were never pure with regard to funding and how maybe corporate funding acts as a counterweight to the historical funding by governments and religious organizations, who never shied about throwing their weight around. But what about when corporations and government collude to control higher education? We are seeing that now.
Professor Kovacic introduces the data access issue:
The Federal Trade Commission, beginning in the late 1970s, began a program to evaluate the effects of past [antitrust] cases. They did this by approaching academic institutions to enlist researchers. With the assistance of a variety of leading academics at the time they spotted researchers, a number of them well known to this group. I was the government's contracting officer representative for the analysis Tim Bresnahan did in the Xerox case.
They paid minimum amounts, barely 13th amendment compliant wages, for them to do the work, but the real inducement to do the work was that they got access to the government's records. And, with some restrictions on confidentiality, it attracted a wonderful group of researchers who did assessments, evaluations.
Professor Valletti mentions during the panel that the most surprising result of their paper was how people reacted to a conflict regarding data, in particular data ownership.
So, sometimes they say that a study was published with data which was public data, publicly available data, and sometimes it was private data. So the data from Uber, in example before. And people react very negatively to private data, irrespective of the direction of the result. This is the surprising thing, not just when you publish an Uber paper, which is positive for Uber — so Uber is great for the community — but also when they author finds with Uber data a result which is not good for Uber. Still there is, of course, less, but it is still a lack of trust by the fact that it is data that is not accessible to other people.
Barrios, et al (2025), says theirs is the first study in economics, to their knowledge, "to quantify the CoI Discount toward academic papers generated by disclosing different conflicts of interest of the authors to the readers."
The paper uses various approaches to assess and measure the impact of various types of conflicts of interest, CoI, such as monetary, career, data access, academic, and political conflicts on "the perceived trustworthiness of economic research." The paper also seeks to support the understanding of "how individuals’ political and economic priors–or personal biases and beliefs–influence both the interpretation of economics research and the potential CoI associated therewith."
Inspired by John, Luigi, Tommaso, and their colleagues, I began thinking about accounting and audit research with these types of conflicts and, in particular, research that uses PCAOB proprietary data. How has the use of PCAOB proprietary data been disclosed over the years? Audit research, in particular, has influenced the PCAOB and audit industry regulation, especially with regard to how PCAOB inspections are used to measure audit quality.
After the paywall, I will apply this insight to PCAOB proprietary data research and the most recent PCAOB inspection results. In particular, I am interested in whether the inspection results can really tell us anything about audit quality, despite what the research may say.
But, first, I want to point out some of the interesting parallels between the paper and accounting and audit research. From Barrios, et al (2025):
One potential criticism of our findings is that they simply reflect a decrease in trust due to perceived conflicts of interest rather than actual ones. Even if this were a mere perception, it would be important. If economists and the public perceive certain results as being less valuable and are less convinced of them, these results are inferior quality products in the marketplace of ideas, becoming less likely to influence public policy and human or corporate behavior. This happens regardless of the correctness of the perception of bias. Nevertheless, we run four “sanity-check” exercises to show that our results are not mere perceptions but are in line with the evidence on the actual biases of conflicted studies.
As we know, the requirement that auditors be independent in both fact and appearance is bedrock in the securities laws and auditing standards. See SEC Chief Accountant Paul Munter in June 2022:
Stated differently, compliance with the prohibitions enumerated in Rule 2-01(c) is necessary but not sufficient. The general standard requires an evaluation of auditor independence, including an assessment of independence both in fact and appearance from the perspective of a reasonable investor. Such a determination cannot be limited to a checklist compliance exercise under Rule 2-01(c). To reiterate: the general standard of Rule 2-01(b) is the heart of the Commission’s auditor independence rule, it always applies, and the Commission investigates and enforces against violations of the general standard.[16]
Professor Valletti also highlights a very interesting observation the researchers made based on manually going through five years of publications in the AER, American Economic Review, and RAND Journal, two prestigious and very well- known economics journals.
AER in a sense, when it comes to conflicts of interest, is the gold standard, and has a very thorough, very precise disclosure policy that you have to do when you publish in the AER. RAND, by and large, has almost no disclosure policy to speak of, so, very little. We find that empirical political papers, which have become more and more common in the profession, basically 40% of them have some sort of potential conflict. We're not negative about it, but when we went to readme files and tried to see, "Can I get access to the data? Where does it come from? Is it accessible to the public?" [we found] lots of those are gated, which is a potential source of conflict.
Now, does it matter? Our argument is there is something does not work in the following sense.
When we compare, within AER and within RAND, whether the papers which are conflicted and non-conflicted, do they have different impact, are they cited more or are they cited less? They are cited equally. These two journals, which are at the opposite ends of the conflict of the disclosure policies, cite conflicted and the non-conflicted equally.
A benign way of interpreting this is that editors are doing a great job, okay, even if RAND doesn't have a disclosure policy, but the editor is aware of it and then they select referees, et cetera, to the extent that they manage to produce equally a valuable research, based on equal citations. But it is something that doesn't work because then the ultimate reader of the paper doesn't perceive that process and people discount the conflict by a lot. If that was the story- that editors internalized everything- people should not react to the conflict of interests of papers which had been published in the same journal at the top, et cetera.
It would be interesting to do the citation analysis on accounting and audit papers. Based on my experience, there is rigidity about which papers must be cited when writing about anything related to audit and accounting. Editors and reviewers enforce citations for a body of literature — often their own — regardless of whether the authors agree with the premise, results, or credibility of the papers.
Professor Valletti says that voluntary disclosure is not enough to fix the problem.
Is it gonna be enough? And it's not gonna be enough because you may say you know just disclose people which are with no conflict, they're gonna say people it will be. When we ask and again, we are asking the top people in the profession, do you disclose appropriately? Eighty percent of the people say yes, but when the same people are asked, do you think you peers are disclosing two percent say yes, okay?
What might help? Valetti says the data conflict is pervasive in the economics profession, and that there should be a minimum checklist covering whether the data is gated or not, discloses the procedure for access, and whether the data provider retains rights to review or withhold results.
Section 7.2 of Barrios, et al (2025), discusses the importance of disclosure of the terms of data access.
The conditions under which researchers gain access to such data are often unclear. While some academic institutions and journals have formal rules prohibiting data providers from retaining the right to block publication, it is uncertain whether these rules are consistently followed. The data agreements between the researchers/universities and the data providers are often confidential (many times even to those within the university who are working with the datasets). This lack of transparency undermines trust in the results of papers that rely on proprietary data, as it leaves open the possibility of selective data sharing or result shaping based on data provider preferences.
Confidentiality in data access agreements is a direct threat to the credibility of academic research. Academic journals should not accept papers where data providers have a right to review the paper for reasons other than maintaining the confidentiality of the data. Academic journals should also require that data agreements (with the possible exception of payment terms) be made public as a precondition for the publication of the article. Such a policy would not only enhance transparency but also increase the bargaining power of researchers vis-`a-vis data-supplying institutions, and in doing so, ultimately safeguard the integrity of academic research.
There are many fruitful paths for further research on how these issues pertain to accounting and audit research, especially with regard to proprietary data access. After the jump, I will tell you what I found when I did a quick review of papers published in top academic journals that utilized proprietary PCAOB data and how these findings might influence how we look at the PCAOB inspection process and the scoreboard approach to reporting results.