Reported earnings are a key metric for market participants when considering investment decisions (e.g., Beyer et al. (2010)), reflecting operational fundamentals more than transitory market noise (Penman 2013). The prediction of earnings changes has long been a cornerstone of fundamental analysis. Prior research shows that financial statement variables can predict future earnings (Ou and Penman 1989; Penman and Zhang 2006). Building on this foundation, Chen et al. (2022) set a new academic benchmark by applying ensemble learning, such as Random Forest and Gradient Boosting, to a granular set of over 4,000 distinct financial items. Their models achieved an Area Under the Curve (AUC) value of 68.66%, significantly outperforming traditional linear regressions. While their work identifies which financial variables carry the most information, their models exclude unstructured textual content, leaving open the question of whether narrative disclosures can further enhance predictive performance.

While quantitative data provides the ‘what,’ narrative disclosures in 10-K filings provide the ‘why,’ offering interpretive context and forward-looking insights into management’s expectations. However, the practical utility of these narratives is under scrutiny. Dyer et al. (2017) provide a comprehensive evolutionary analysis of 10-K filings from 1996 to 2013, documenting an increase in document length driven largely by regulatory requirements. Crucially for practitioners, they found that this expansion often results in declining readability and an increase in boilerplate language, that is, repetitive text that may obscure rather than reveal firm-specific risks. While narratives like the Management’s Discussion and Analysis (MD&A) can signal future performance (Brown and Tucker 2011), they can also be used strategically to obfuscate poor results through linguistic complexity (Li 2008). For investors and analysts, the primary challenge lies in distinguishing between genuine economic signals and regulatory noise. Our study addresses this by testing whether automated tools can effectively filter this noise to enhance prediction.

Textual analysis has become central in accounting and finance as narrative disclosures complement traditional financial information. To process the thousands of pages in a typical 10-K corpus, accounting research has shifted from simple word-counting to sophisticated Natural Language Processing (NLP). Early studies focus on readability (Li 2008) and sentiment (Loughran and McDonald 2011), offering simple textual measures but limited semantic depth. More recent NLP tools, including transformer-based models like FinBERT (Huang et al. 2023), capture richer narrative patterns, though their complexity and high dimensionality make the underlying drivers of prediction difficult to interpret. In this context, topic modeling, particularly LDA, provides a useful middle ground. LDA represents each document as a mixture of latent topics, allowing users to identify the specific themes (e.g., ‘Renewable energy’ or ‘Corporate governance’) being discussed. Each topic is characterized by a distribution over words, producing interpretable topic proportions that can be readily incorporated into predictive models.

LDA has been widely applied to various financial texts. For example, Campbell et al. (2014) extract topics from risk disclosures in 10-K filings; Huang et al. (2018) compare analyst reports with earnings calls, and Feuerriegel et al. (2016) study market reactions to news announcements. Beyond those analyses, several studies have used topic distributions as predictive variables. Hanley and Hoberg (2010) incorporate topics into regressions predicting IPO underpricing, and Brown et al. (2020) use topic proportions from 10-K filings to predict financial misreporting. For practitioners, LDA offers an automated way to analyze thousands of filings simultaneously, identifying thematic shifts. By integrating these LDA-derived topics with the financial benchmark established by Chen et al. (2022), we provide a test of whether automated textual analysis truly adds value in the context of short-horizon earnings prediction.

The sample consists of S&P 500 firms between 2013 and 2023 for three reasons:

1. the index captures roughly 80% of U.S. equity market capitalization (S&P Global 2025);
2. these firms follow strict reporting standards, producing documents that are standardized yet retain firm-specific narrative content, making them suitable for topic modeling (Li 2010); and
3. the sample size balances representativeness, computational feasibility, and statistical power.

To ensure a consistent definition of the S&P 500 universe throughout the sample period, we use constituents as of February 2024. Pre-processed 10-K filings are obtained from the Notre Dame Software Repository¹ (accessed: May 18, 2025). After matching filings to the S&P 500 constituents using Central Index Keys (CIK), the sample includes 5,250 documents.

Building upon the one-stage parsed text files from the repository, we apply additional preprocessing steps to optimize the corpus for topic modeling. First, paragraphs shorter than 80 characters or containing over 50% non-alphabetic characters are removed. URLs and emails are eliminated using regular expressions, and the text is tokenized into individual words with English stopwords filtered out. Words shorter than four characters and domain-specific terms such as ‘fiscal’ or ‘quarterly’ are also removed. A document-term matrix (DTM) is then constructed and dimensionality reduced by excluding terms appearing in fewer than 0.1% or more than 80% of documents (Dyer et al. 2017). This approach preserves terms with potential topical significance while minimizing noise, forming the foundation for subsequent LDA topic modeling. After text preprocessing, we obtain a document-term matrix of 5,241 10-K filings and 87,106 unique terms.

Then, we extract specific sections from 10-K filings including: Management’s Discussion and Analysis (MD&A, typically Item 7), Risk Factors (Item 1A), Controls and Procedures (Item 9A), Changes in Accounting and Disagreements (Item 9), Unresolved Staff Comments (Item 1B), and Quantitative and Qualitative Disclosures (Item 7A). The section extraction algorithm employs a multi-step approach. It first scans the document for standard SEC item headers using pattern recognition to identify section boundaries. Next, it extracts the text content between identified headers, creating separate text segments for each section. Finally, the same preprocessing procedures used for the main corpus (see previous paragraph) are applied to each section, ensuring consistency in tokenization, stop-word removal, and term filtering.

We obtain annual financial data from Compustat for 2009–2024, a timeframe necessary to calculate earnings trend components while remaining consistent with the 10-K sample. To construct the dependent variable for earnings prediction, we follow Chen et al. (2022) and focus on earnings per share excluding extraordinary items (EPSPX). We then define a binary indicator equal to 1 if a firm’s EPS increases and 0 if it decreases from year t to year t + 1. The EPS change is detrended by subtracting a drift term, calculated as the average EPS change over the previous four years to control for firm-specific growth trends and enable meaningful cross-sectional comparisons.

We calculate a set of 21 financial variables identified in prior literature as significant predictors of earnings changes. These variables span five key dimensions of financial performance: profitability, liquidity, leverage, efficiency, and market valuation. Ou and Penman (1989) evaluate 68 financial statement items and identify 16 indicators that collectively predict earnings direction more accurately than price-based measures alone. Sloan (1996) highlights the differential persistence of earnings components, showing that firms with high accrual components typically experience subsequent earnings reversals, while cash flow components persist more reliably. This explains why we include both accrual-based and operating cash flow measures to capture the varying information content of earnings components. Piotroski (2000) develops the F-Score using nine fundamental signals across profitability (ROA, CFO, change in ROA, accruals), leverage/liquidity (change in debt ratio, change in current ratio, equity offerings), and operating efficiency (change in gross margin, change in asset turnover). Instead of calculating the composite F-Score, we include its component ratios individually to allow differential weighting in the predictive model.

For a subset of variables where a missing value corresponds to the absence of the underlying activity, we replace missing entries with zero. For instance, this applies to inventory (INVT), capital expenditures (CAPX), or depreciation and amortization (DP). For other variables, missing values are imputed using industry averages based on the Fama-French 48 industry classification, constructed from each firm’s SIC code (see Chen and McCoy (2024)). Of all firm-year observations, 1,759 require imputation using the industry mean, with 1,322 cases due to missing inventory turnover. While this accounts for about 33% of observations, it affects only 4% of the underlying data entries in the dataset. Table 1 presents the definitions and summary statistics of our financial variables.

To identify the main themes discussed in the 10-K filings, we apply LDA, a widely used unsupervised machine-learning technique (Blei et al. 2003). LDA treats each filing as a mixture of underlying topics and each topic as a collection of words that tend to appear together. The key decision in LDA is choosing the number of topics (K). Too few topics merge distinct ideas into overly broad categories; too many create specific or hard-to-interpret themes.

We evaluate five candidate values K ∈ {20, 60, 100, 140, 180} covering a range from coarse to highly granular topic structures, using two complementary criteria. Perplexity provides a measure of model fit, with lower values indicating better out-of-sample performance, but it does not guarantee that topics are meaningful. As shown by Mimno et al. (2011), models with strong perplexity can still produce semantically incoherent topics. To address this limitation, we complement perplexity with topic coherence (Röder et al. 2015), which evaluates how consistently a topic’s top words co-occur in the corpus and better reflects human interpretability. As shown in Figure 1, perplexity decreases sharply at lower values of K and then levels off between 140 and 180. Coherence scores, however, continue to rise over this interval, indicating greater interpretability without clear signs of overfitting. We therefore select K = 140 as the optimal number of topics. This choice aligns with prior work, including Dyer et al. (2017), who use 150 topics.

Figure 1.

Selection of optimal K. Notes: This figure presents the perplexity and coherence scores for different numbers of topics. Perplexity measures the model’s predictive performance as follows:$\mathrm{Perplexity}(D)=\exp \{-\frac{\sum _{d=1}^M\log p\left(w_d\right)}{\sum _{d=1}^M{N}_d}\}$ where p(w_d) denotes the document likelihood under the fitted model and N_d is the number of words in each document. We train the model on 75% of the data and then calculate the perplexity using a random hold-out sample of the remaining 25% of the observations. The coherence score is computed as follows:$C_v(T)=\frac{1}{N}\sum _{i=2}^N\sum _{j=1}^{i-1}\log \frac{P(w_i,w_j)+ϵ}{P\left(w_i\right)P\left(w_j\right)}$ where P(w_i, w_j) denotes the probability of terms co-occurring within a sliding window (default = 110 words) in the 10-K corpus, and 𝜖 is a smoothing constant. In a second stage, we narrow the search to K ∈ {140, 150, 160, 170, 180} to capture potential marginal gains. Untabulated results highlight that coherence increases further but only modestly. After fixing the number of topics to 140, we estimate the final LDA model using Gibbs sampling with 1,000 iterations and a burn-in of 250 to ensure convergence. Hyperparameters α and β follow standard symmetric values of 50/K and 0.1, respectively.

The estimation yields two outputs: (1) the document-topic distribution matrix, which provides each 10-K’s probability distribution over the 140 topics and serves as textual features in the earnings prediction models, and (2) the top words for each topic, enabling qualitative interpretation. At this stage, we exclude filings with fewer than 100 characters after preprocessing. The final dataset comprises 5,155 firm-year observations including 495 companies over the period 2013–2023.

The empirical results provide mixed evidence on the incremental predictive power of narratives from 10-K filings for predicting earnings changes. Table 2 presents the performance metrics for all six model configurations. XGBoost outperforms both Lasso regression and Random Forest, with the baseline model achieving the highest performance across all metrics: AUC = 0.750, Accuracy = 0.695, and F1 Score = 0.699. This finding is consistent with recent machine learning literature emphasizing the superior performance of gradient boosting methods for structured prediction tasks (Chen et al. 2022).

Contrary to expectations, the enhanced models, incorporating 140 LDA-derived topic features alongside financial variables, generally exhibit slightly lower performance than the baseline models using only financial variables. This pattern is consistent across AUC and Accuracy metrics, where enhanced models underperform baseline models. Specifically, the AUC (Accuracy) declines from 0.676 to 0.654 (0.623 to 0.592) for Lasso, from 0.733 to 0.714 (0.667 to 0.664) for Random Forest and from 0.750 to 0.737 (0.695 to 0.672) for XGBoost. The F1 Score metric presents a more nuanced picture, showing minor improvements for Lasso and Random Forest enhanced models (0.651 to 0.652 and 0.667 to 0.674, respectively). Overall, these findings suggest that the relationship between narratives and earnings changes is complex. While incorporating LDA-derived topic features can yield modest gains on a specific metric, using 140 topics does not provide systematic incremental predictive power beyond traditional financial variables.

We then conduct two untabulated robustness tests. First, we examine whether including COVID-19 years influences model performance. We re-estimate the models using 2013–2019 as the training period and 2023 as the test period, thereby excluding 2020–2022. The results are consistent with the main findings, with baseline models outperforming their enhanced counterparts across all three algorithms. Second, we investigate whether the high dimensionality of LDA explains the lack of incremental predictive value. Reducing the number of topics from 140 to 100 yields mixed results. Random Forest exhibits improved prediction with 100 topics, whereas XGBoost continues to underperform, and Lasso remains largely unchanged.

We examine feature importance when using the enhanced XGBoost model. Figure 2 reports the top 20 features. Traditional financial ratios – particularly price-to-earnings ratio, ROA, accruals, and gross margin – dominate the ranking, while the most important topic features exhibit low importance. This result is consistent with prior earnings-prediction research (Ou and Penman 1989; Chen et al. 2022) and highlights the continued relevance of structured financial information relative to narrative disclosures.

Figure 2.

XGBoost – Top 20 most important features. Notes: The different scales on the horizontal axis reflect distinct feature importance. XGBoost uses relative gain (0–1 scale).

To evaluate the contribution of narratives, we further inspect the top 40 topic features from the XGBoost model with 140 topics in Figure 3. Since LDA provides word distributions rather than semantic labels, topic interpretation relies on qualitative assessment of each topic’s most important words. The examination of topic composition reveals several meaningful patterns that contribute to the prediction of earnings changes.

Figure 3.

XGBoost – Top 40 most important topic features. Notes: The different scales on the horizontal axis reflects distinct feature importance. XGBoost uses relative gain (0–1 scale).

A first pattern is that many influential topics capture industry-specific disclosure language. Examples include topics related to healthcare and pharmaceuticals (Topics 118, 36, 70, 61, 108), energy and utilities (Topics 62, 87, 93, 40, 111), and technology-related themes (Topics 81, 7, 127). For instance, words in Topic 118 include ‘fertility’, ‘kidney’, ‘capitation’, and ‘practitioner’ and words in Topic 62 include ‘liquefaction’, ‘compressor’, and ‘energy’. This suggests that industry-specific linguistic patterns may embed information relevant for predicting earnings changes. Within these industry-oriented patterns, some topics further capture detailed elements of firms’ business models and operations. For instance, Topic 78 (Retail & Product Assortment) includes keywords such as ‘assortment’, ‘markdown’, and ‘ecommerce’, pointing to retail merchandising and store-format adjustments. Topic 44 (Real Estate & Land Development) features ‘tract’, ‘propco’, and ‘plat’, reflecting land development and property-holding structures. Topic 87 (Renewable Energy Technology) is defined by technical terms such as ‘microinverter’, ‘inverter’, and ‘connector’, capturing solar-energy hardware disclosures. These operational themes suggest that LDA extracts variation in firms’ activities that is not fully captured by financial variables. Second, beyond industry-specific themes, several topics reflect functional aspects of complex financial and governance practices. For instance, Topic 106 (Financial Instruments) encompasses specialized banking and regulatory terminology, including words such as ‘noncumulative’ (non-cumulative preferred shares), ‘tlac’ (Total Loss-Absorbing Capacity requirements), and ‘hqla’ (High-Quality Liquid Assets). Topic 49 (Corporate Governance) contains procedural language related to shareholder meetings and securities administration, such as ‘adjourn’, ‘securityholder’ and ‘CUSIP’.

We conduct a section-level analysis to examine how the most important topics are distributed across different sections of the 10-K filings. After identifying each section, we compute the average term frequency for every section type across all filings in the sample. To assess the predictive relevance of each section, we compute cosine similarity scores between the 40 most important topics identified through XGBoost and the term distributions of each 10-K section. This approach allows us to map topics to their most likely source sections without re-estimating separate topic models for individual sections, thereby maintaining consistency with the main analysis.

Figure 4 presents the distribution of important topics across 10-K sections. The results reveal that Management’s Discussion and Analysis contains the highest concentration of predictive topics, capturing more than 25 of the 40 most important topics. The next most influential sections are Risk Factors (7 topics) and Controls and Procedures (4 topics).

Figure 4.

Number of important topics associated with each 10-K section. Notes: The horizontal axis reflects the number of important topics, obtained with XGBoost, associated with each 10-K section.

For practitioners, the implications are threefold:

1. The targeted use of narrative disclosures may still provide qualitative value. The higher relevance of certain sections suggests that analysts should focus selectively on these portions rather than ingesting entire filings. Such prioritization can enhance efficiency while preserving the most decision-relevant insights.
2. Our findings highlight important caveats regarding automation. Automated textual features can introduce noise and methodological sensitivity – for example, through choices related to topic modeling parameters or preprocessing techniques. Practitioners should rigorously validate any text-augmented model using out-of-sample testing and benchmark its performance against traditional approaches using economically meaningful metrics. This caution extends to the broader adoption of modern AI systems, which often function as black boxes and require careful scrutiny before integration into decision-making processes.
3. Corporate narratives frequently echo quantitative disclosures or contain low-information content. As a result, for most investment and valuation workflows, the marginal benefit of systematically incorporating narrative data appears limited. In many cases, these benefits may not justify the additional complexity, implementation challenges, and data-processing costs involved.

Conceptually, the results suggest an information redundancy interpretation: narrative disclosures in 10-K filings may largely echo information already incorporated in traditional financial metrics. This interpretation is consistent with prior evidence that narrative disclosures often contain boilerplate and low-information content (Dyer et al. 2017), and with the idea that management may strategically obfuscate poor performance by increasing textual complexity (Li 2008). Predictive information appears to be localized in specific sections, and whole-document analysis may dilute these concentrated signals, consistent with the evidence in Campbell et al. (2014). From a practical perspective, our findings provide regulators, such as the SEC, with empirical guidance on which sections of the 10-K might convey information most relevant for investors, potentially motivating refinements to disclosure standards that discourage excessive boilerplate while reinforcing economically meaningful content.

At the same time, the mixed predictive results highlight the instability and limited robustness of textual features in earnings prediction in this context. Whether these results generalize to non-US settings remains an open question. On one hand, the structured nature of US disclosures makes narrative information easier to compare and analyze. On the other hand, less structured reporting in other settings may provide firms with greater opportunities to disclose unique, non-boilerplate information. Model performance is sensitive to methodological choices, particularly the number of topics, reflecting well-established concerns over topic-modeling instability and the challenges of extracting reliable signals from high-dimensional text (Greene et al. 2014; Lewis and Young 2019). While dimension reduction techniques can improve performance for specific algorithms, the overall pattern suggests that the incremental value of LDA-derived features remains modest. Alternatively, the limited incremental value of textual features may reflect a temporal mismatch: narratives could be more informative for multi-year horizons rather than immediate annual earnings changes. Further research could disentangle between the two explanations.

Our study contains several limitations. First, the limited predictive value of LDA-derived topics partly reflects methodological constraints inherent in standard unsupervised topic modeling. The bag-of-words representation disregards word order and syntax, leading to information loss when converting qualitative text into quantitative features (Loughran and McDonald 2016). LDA assumes that each document is a mixture of latent topics optimized for textual coherence rather than predictive accuracy (Taddy 2013), which may cause misalignment with financially relevant information. Future work could explore transformer-based models (e.g., BERT or FinBERT) to better capture the semantic and rhetorical structure of narrative disclosures. Second, many extracted topics resemble industry labels dominated by sector-specific jargon, suggesting that LDA predominantly captures broad industry traits rather than firm-level signals. This dilution is especially pronounced in cross-industry samples, highlighting the potential value of industry-specific topic modeling. Third, topic modeling remains highly sensitive to the number of topics. Despite using grid search and perplexity-coherence metrics, topic interpretability and predictive performance vary substantially across specifications, illustrating trade-offs between interpretability, granularity, and model fit. These limitations suggest that more nuanced approaches, such as supervised LDA (Gentzkow et al. 2019), embedding-based representations (Huang et al. 2023), or stability-enhancing methods (Greene et al. 2014), may better capture the economic substance of narrative disclosures.