Generative Economic Research

Academic Research: Korinek 2023

Author: Daniel Seiler

Date: February 20, 2024

In this blog, I would like to unpack the revelations from Anton Korinek's seminal work, "Generative AI for Economic Research: Use Cases and Implications for Economists." As we navigate the complexities of asset management, Korinek's foray into the realm of generative AI and large language models (LLMs) such as ChatGPT offers an enlightening and transformative perspective. While Korinek concentrates on AI's role in economic research, there are numerous parallels to asset management that can be drawn.

What are the intersting findings?

The parallels between economic research and asset management are striking and immediate. Korinek's revelation that generative AI is a game-changer in productivity, by taking over micro-tasks, is a beacon of advancement for asset managers. We're talking about a revolution in how we generate ideas, communicate findings, and dissect data.

Korinek's rich insights weaves through six pivotal realms where LLMs excel: ideation, writing, research, data analysis, coding, and the delicate art of mathematical derivations. His analysis is thorough, assessing the efficacy of LLMs across each of these categories.

The table below delves deeper into Korinek's nuanced narrative. These AI models are poised to significantly enhance the ideation and writing processes, transforming them into an almost autonomous flow. However, there is an undercurrent of caution: the accuracy of literature research and the precision required in mathematical derivations remain areas where AI still falters. Korinek offers a candid assessment of these challenges, especially highlighting the tendency of LLMs to "hallucinate" data, serving as a reminder of the obstacles that lie ahead.

Yet, it's not just a story of limitations. Korinek's work is a call to the potential that lies within generative AI.

---

Summary of LLM Capabilities and Rating of Usefulness

Category		Task		Usefulness
Ideation and Feedback		Brainstorming		●
		Feedback		◐
		Providing counterarguments		◐
Writing		Synthesizing text		●
		Editing text		●
		Evaluating text		●
		Generating catchy titles & headlines		●
		Generating tweets to promote a paper		●
Background Research		Summarizing Text		●
		Literature Research		○
		Formatting References		●
		Explaining Concepts		◐
Coding		Writing code		◐
		Explaining code		◐
		Translating code		●
		Debugging code		◐
Data Analysis		Creating figures		◐
		Extracting data from text		●
		Reformatting data		●
		Classifying and scoring text		◐
		Extracting sentiment		◐
		Simulating human subjects		◐
Math		Setting up models		◐
		Deriving equations		○
		Explaining models		◐

---

Notes — The third column reports Korinek's subjective rating of LLM capabilities as of September 2023:
○ Experimental; results are inconsistent and require significant human oversight.
◐ Useful; requires oversight but will likely save you time.
● Highly useful; incorporating this into your workflow will save you time.

Why is it important for us?

As asset managers, we're on the cusp of an era where investment processes could be revolutionized by AI—an age where automated bots don't just assist but lead the charge.

Translating the categories from Korinek's assessment into various steps of an investment process, we arrive at a structure akin to "idea generation" (Ideation and Feedback), "research" (Background Research, Coding, Data Analysis, Math), and "communication" (Writing). Korinek's observations suggest that LLMs could significantly impact the stages of idea generation and communication. It's increasingly conceivable that these elements of the investment process could soon be fully automated and managed by AI bots.

Interestingly, Korinek points out the challenges in literature research. While it may seem difficult for LLMs, he suggests that narrowing the task to finding the closest article to an existing one could be within their capabilities. However, he expresses quality concerns with current LLMs, particularly when searching and referencing academic literature. The tendency of LLMs to "hallucinate" or generate non-existent, authoritative-sounding papers is especially problematic in heavily regulated sectors like investment. Nonetheless, from our experience, these challenges can be completely mitigated with thorough checks and the development of new techniques.

When it comes to creative tasks such as "deriving equations," the issue is not hallucination but accuracy. LLMs are prone to errors in mathematical derivations, which is unacceptable in the exacting field of investing. As a result, while LLMs are promising in aiding the investment process, their role in implementing investment decisions is still limited. Thankfully, the field of quantitative finance offers considerable support in this area.

In conclusion, Korinek's assessment highlights the enormous potential of LLMs within the asset management industry. While they are currently a supportive tool, the rapid evolution of these models suggests a future where automated investment bots could become commonplace.

At Eqitron, we are at the forefront, actively exploring and integrating these cutting-edge advancements. We're not just observers; we're pioneers, eagerly translating Korinek's findings from theory to practice, from potential to tangible results in asset management.