Generative Economic Research

Academic Research: Korinek 2023

Author: Daniel Seiler

Date: February 20, 2024

In this blog, I would like to unpack the revelations from Anton Korinek's seminal work, "Generative AI for Economic Research: Use Cases and Implications for Economists." As we navigate the complexities of asset management, Korinek's foray into the realm of generative AI and large language models (LLMs) such as ChatGPT offers an enlightening and transformative perspective. While Korinek concentrates on AI's role in economic research, there are numerous parallels to asset management that can be drawn.

What are the intersting findings?

The parallels between economic research and asset management are striking and immediate. Korinek's revelation that generative AI is a game-changer in productivity, by taking over micro-tasks, is a beacon of advancement for asset managers. We're talking about a revolution in how we generate ideas, communicate findings, and dissect data.

Korinek's rich insights weaves through six pivotal realms where LLMs excel: ideation, writing, research, data analysis, coding, and the delicate art of mathematical derivations. His analysis is thorough, assessing the efficacy of LLMs across each of these categories.

The table below delves deeper into Korinek's nuanced narrative. These AI models are poised to significantly enhance the ideation and writing processes, transforming them into an almost autonomous flow. However, there is an undercurrent of caution: the accuracy of literature research and the precision required in mathematical derivations remain areas where AI still falters. Korinek offers a candid assessment of these challenges, especially highlighting the tendency of LLMs to "hallucinate" data, serving as a reminder of the obstacles that lie ahead.

Yet, it's not just a story of limitations. Korinek's work is a call to the potential that lies within generative AI.

Summary of LLM Capabilities and Rating of Usefulness
Category Task Usefulness
Ideation and Feedback Brainstorming
Providing counterarguments
Writing Synthesizing text
Editing text
Evaluating text
Generating catchy titles & headlines
Generating tweets to promote a paper
Background Research Summarizing Text
Literature Research
Formatting References
Explaining Concepts
Coding Writing code
Explaining code
Translating code
Debugging code
Data Analysis Creating figures
Extracting data from text
Reformatting data
Classifying and scoring text
Extracting sentiment
Simulating human subjects
Math Setting up models
Deriving equations
Explaining models

    Notes — The third column reports Korinek's subjective rating of LLM capabilities as of September 2023:
    ○ Experimental; results are inconsistent and require significant human oversight.
    ◐ Useful; requires oversight but will likely save you time.
    ● Highly useful; incorporating this into your workflow will save you time.

Why is it important for us?

As asset managers, we're on the cusp of an era where investment processes could be revolutionized by AI—an age where automated bots don't just assist but lead the charge.

Translating the categories from Korinek's assessment into various steps of an investment process, we arrive at a structure akin to "idea generation" (Ideation and Feedback), "research" (Background Research, Coding, Data Analysis, Math), and "communication" (Writing). Korinek's observations suggest that LLMs could significantly impact the stages of idea generation and communication. It's increasingly conceivable that these elements of the investment process could soon be fully automated and managed by AI bots.

Interestingly, Korinek points out the challenges in literature research. While it may seem difficult for LLMs, he suggests that narrowing the task to finding the closest article to an existing one could be within their capabilities. However, he expresses quality concerns with current LLMs, particularly when searching and referencing academic literature. The tendency of LLMs to "hallucinate" or generate non-existent, authoritative-sounding papers is especially problematic in heavily regulated sectors like investment. Nonetheless, from our experience, these challenges can be completely mitigated with thorough checks and the development of new techniques.

When it comes to creative tasks such as "deriving equations," the issue is not hallucination but accuracy. LLMs are prone to errors in mathematical derivations, which is unacceptable in the exacting field of investing. As a result, while LLMs are promising in aiding the investment process, their role in implementing investment decisions is still limited. Thankfully, the field of quantitative finance offers considerable support in this area.

In conclusion, Korinek's assessment highlights the enormous potential of LLMs within the asset management industry. While they are currently a supportive tool, the rapid evolution of these models suggests a future where automated investment bots could become commonplace.

At Eqitron, we are at the forefront, actively exploring and integrating these cutting-edge advancements. We're not just observers; we're pioneers, eagerly translating Korinek's findings from theory to practice, from potential to tangible results in asset management.