Comprehensive assessment of firm financial performance using financial ratios and linguistic analysis of annual reports

Indicators of financial performance, especially financial ratio analysis, have become important financial decision-support information used by firm management and other stakeholders to assess financial stability and growth potential. However, additional information may be hidden in management communication. The article deals with the analysis of the annual reports of U.S. firms from both points of view, a financial one based on a set of financial ratios, and a linguistic one based on the analysis of other information presented by firms in their annual reports. Spearman correlation coefficient is used to compare the values of financial and linguistic indicators. For the purpose of the comprehensive assessment, novel word lists are proposed, specifically designed for each category of financial analysis. The aim is to assess the information ability of annual reports and whether successful firms present their results precisely or not. The results show that the proposed topic dictionaries can be beneficial, especially for the assessment of cash flow and leverage ratios.


INTRODUCTION
Financial stability of a firm is associated with its ability to generate profit, increase the value of invested capital and at the same time repay its short-and long-term liabilities.Assessment of financial performance is primarily based on various methods of financial analysis.The choice of methods is mainly influenced by the purpose of use, time criteria, character of information resources or the degree of algorithm development.The aim is to achieve the desired level of complexity in evaluating firm and its activities.
In the practise of financial analysis, financial ratios are mainly used for their simplicity and additional information value.These ratios make it possible to analyze the evolution of the financial situation of a firm (trend analysis), cross-sectional analysis and comparative analysis.Financial ratios can be categorized into the indicators of productivity, profitability, cost, liquidity, solvency, capital structure, and capital market indicators.Financial ratios are the most popular and most widely used methods of financial analysis also because they can be used as input data of more complex mathematical models.On the other hand, some authors prefer purposeful selection of indicators, for example, as in DuPont decomposition (Burns et al., 2008), creditworthiness (diagnostic) models or bankruptcy (predictive) models (Beaver et al., 2010;Apergis et al., 2011).Predictive models based on the real-life financial data are considered essential, such as Altman model and its modifications as well as Ohlson model.
To assess the results and to predict future financial development of a firm it is necessary to connect data from financial analysis and other information that the firm itself presents mainly in its annual report.This is mainly a verbal analysis of the causes that led to the attainment of positive or negative financial results.Annual reports also present company's managerial priorities.
Assuming that firm management monitors all aspects of financial health through financial analysis, a set of indicators cannot only be chosen to evaluate financial performance, but it should also be monitored and assessed whether the results achieved are commented and explained in the textual sections of annual reports.The aim of this article is to assess the information ability of firm annual reports in terms of comments on financial performance using both existing and novel dictionaries proposed by the authors.The novelty of this approach lies in the proposal of several dictionaries specifically for financial analysis.In addition, it is examined how the frequency of words related to financial analysis correlates with the firms' financial performance in terms of financial ratios.The performance of these dictionaries is compared with those proposed by Loughran and McDonald (2011) and demonstrate that our novel dictionaries performed better for liquidity and leverage ratios, in particular.
The remainder of this paper has been organized as follows.The next section reviews the related literature.Next, research methodology is introduced.Section 3 describes the process of data collection and their descriptive statistics.Section 4 examines the correlations between the proposed word lists and financial indicators.Our conclusions and future research directions are drawn in the final section.

LITERATURE REVIEW
The potential of financial analysis in assessing the financial health of the firm and its performance has attracted considerable attention in recent literature (e.g.Kotane & Kuzmina-Merlino, 2012;Beaver et al., 2010;Kovárík & Klímek, 2012;Brendea, 2014;Lee, 2014;Kubenka, 2016).However, the linguistic evaluation of firm activities has been increasingly important for the overall evaluation of financial performance due to the loss of informative value of financial indicators (Beaver et al., 2005).Moreover, the comments provided by the firm management are also important for the correct interpretation of financial results.
Previous research on textual analysis of company related texts was surveyed by Kearney and Liu (2014), Nassirtoussi et al. (2014) and Loughran and McDonald (2016).Significant differences of word categories has been observed for firms with low/high earnings and stock returns (Li, 2008), stock market volatility (Loughran & McDonald, 2011), market-to-book ratio (Myskova & Hajek, 2016), return on assets (Davis et al, 2012), credit ratings (Hajek & Olej, 2013), Altman Z-score (Hajek et al., 2014).However, the dictionaries used previously are mostly limited with the focus on general linguistic categories, particularly on positive and negative sentiment.Dictionaries focused on specific areas of financial decision-making has been used only rarely (Myskova & Hajek, 2015).

METHODOLOGY
To achieve the objective defined in the previous section, the following hypothesis can be posed: the management of the firm with better financial results comments the firm's financial performance in their annual reports in more detail compared with those performing financially worse.To test this hypothesis, the following research methodology was used: 1. Create a subset of financial ratios and perform the financial analysis; 2. Develop dictionaries (word lists) to assess the scope and structure of managerial comments to the financial ratios in annual reports, 3. Apply the novel dictionaries, together with existing dictionaries, to a dataset of annual reports test the hypothesis.The analysis was performed using Spearman correlation coefficient due to the absence of normal distribution for financial and linguistic variables.The source of information for financial analysis was a set of annual reports for 1380 U.S. firms listed in major U.S. stock exchanges for the year 2013 (www.sec.gov/edgar.shtml).In the year 2013, major U.S. stock exchanges returned to growth.Investor sentiment was also more optimistic compared with previous years.This trend has remained to the present day.Therefore, it can be expected that with better financial performance, the managerial comments will also be more optimistic and more focused on financial ratios.On the other hand, the results for more recent years should be analysed with caution as managers begin to be aware of the importance of the sentiment and content of their comments on stakeholders' behaviour.
Financial analysis includes the selection of appropriate financial ratios and their calculation for the reporting period.The overall financial evaluation was based on the following points of view: 1. operating -the ratios of profitability, capital return and capital turnover -to predict and plan future financial performance, 2. investment -indicators to determine firm attractiveness for investors, 3. financial structure and solvency -indicators evaluating firm structure in terms of ability to meet obligations -short-term (liquidity) and long-term (gearing).The problem in assessing the indicators of profitability may be the use of different profit modifications.Preference in expressing earnings can be characterized as follows.Earnings After Taxes

Renáta Myšková, Petr Hájek
Comprehensive assessment of firm financial performance using financial ratios and …

99
(EAT) is important from the perspective of the owner because it represents a profit to be distributed.On the other hand, Earnings Before Taxes (EBT) is preferable in terms of comparability of firms from countries with different taxation or changes in tax rates over time.When accentuating the growth in revenue and cost control, Earnings Before Interest and Taxes (EBIT) is usually applied.Return on Equity (ROE), Return on Assets (ROA) and Return on Sales (ROS) were selected to represent profitability ratios.
The ROE is widely used in practice although it is sometimes criticized for not taking into account the problem of risk associated with business activities and the size of the initial capital invested or future income (Brigham & Houston, 2006;Parrino & Kidwell, 2009), so it is difficult to correctly assess the effect on shareholder value.ROE value (usually expressed in percentage) should be higher than the interest at which the firm can borrow external capital.ROA is the ratio of EBIT to total assets invested in the business.It allows us to compare firms with different proportions of debt in financial resources.
According to the theory of corporate finance, the value of ROA can be assessed as follows: > 15% denotes a very good position of the firm, 12%-15% a good position, 8%-12% a moderate position, < 8% a poor position, and at 0% is the firm's existence at risk.ROS reflects the firm's profit margin and is calculated with EBIT because it eliminates the effect of various capital structures and possibly a different level of taxation.Activity indicators were represented by Total Assets Turnover Ratio.The value of this ratio greater than 1 is usually deemed acceptable, while the optimum recommended value is 1.5.Firms with lower values of this ratio should consider the reduction of assets.
For the purpose of market evaluation, indicators were selected as follows: price to equity (P/E), market price to book value (PBV) (also referred as Market-to-Book Ratio), and payout ratio.When assessing the optimal level of P/E, the relatively high value of the indicator signals the possibility of a larger dividend growth, whereas lower value of the P/E may indicate a greater risk or low growth potential.The advantage of the PBV is its focus on multiple components of equity, not only on net income.The PBV indicator should be higher than 1.
Payout ratio is basically a market indicator, but it can significantly boost investor interest because it shows what portion of the net income is paid out to shareholders in the form of dividends.
The financial structure of the firm was assessed in terms of liquidity.Given that the liquidity of the firm is related to a certain amount of relatively free capital, the indicator of Net Working Capital (NWC) was monitored relatively to total assets.Liquidity was also assessed with respect to cash to total assets ratio (the portion of a firm's assets held in cash or marketable securities).
Other selected cash flow indicators were free cash flow to the firm (FCFF) and free cash flow to the equity (FCFE).The fact that the company has the necessary capital to invest can be assessed, among others, in the form of retained earnings for further development.Therefore, the ratio of retained earnings to total assets (RetEar/TA) was monitored.
Total leverage (also referred to as an indicator of credit risk) is determined by the ratio of total debt to total assets of the firm.The average value of the total debt should reach 0.3 to 0.5.Lower values are considered ineffective, while values above 0.7 are deemed risky.Book value of debt includes short-term and long-term liabilities, but trade payables and accrued liabilities are not taken into account.
The long-term debt was also investigated as a ratio of long-term debt to equity and financial leverage.In the U.S., financial leverage is monitored in terms of shareholders' equity, so as the ratio of long-term (total) liabilities / shareholder's funds.Generally, the leverage ratio is based on the fact that external capital is usually cheaper than the internal one until the firm is able to increase the value of every dollar of capital more than the interest rate on the debt.The index of financial leverage, this is the ratio of ROE to ROA, should always be greater than 1.
To create dictionaries for assessing the extent and structure of the comments on the financial ratios in annual reports, it was necessary to define a list of keywords for each category of financial indicators.Sample list of these words for the financial analysis is presented in Table 1.It is clear that just as strong correlations can be expected between the indicators of financial analysis, overlaps also exist in the case of the proposed dictionaries.In fact, several words (phrases) are included in multiple categories.From the proposed word lists it is also clear that the purpose of these dictionaries is not to obtain a positive/negative context.Table 1 Word lists for financial analysis (FA)

Category
Word list FA-Profitability profit, earnings, return on, margin, income FA-Activity additions to assets, revaluation of assets, devaluation of assets, capital acquisition, assets acquisition, payback, debtors turnover, debtors days, stock day ration, average collection period, stock turnover, inventory turnover, stock turnover, credit day ratio, debt payment period, debt service coverage , total assets turnover, turnover of fixed assets, asset turnover, commitments turnover, capital productivity, capacity utilization rate, capital turnover, sales per employee, revenue per employee, operating cycle, operational performance, operating performance, days inventory, days sales, days payable, operating revenue to total assets, working capital to sales, sales to total assets, enterprise value to sales FA-Market dividend cover, dividend yield, per share, payout ratio, plowback ratio, sustainable growth rate, market-to-book, price-to-book, ordinary share, preferred share, proposed dividend, share value above par, total return, current share price, price earnings, priceto-earnings, price cash flow, price to cash flow, e.p.s., equity investor, growth investor, value investor, profit attributable to ordinary shareholders, ordinary shares in issue, common shares outstanding, earnings per ordinary share, rate of return, earnings yield, capital gearing ratio, debentures, valuation ratio, earnings growth, stock price, market value, P/E, earnings per share, growth stock, PEG ratio, price sales, price to sales, enterprise value, market capitalization, retained earnings, reinvestment rate, payout ratio, beta FA-Liquidity liquidity, cash position, current ratio, current savings, net working capital, non-cash working capital, cash ratio, quick ratio, acid-test, cash conversion cycle, liquid assets FA-Cash flow cash return, cash value added, cash payments, average collection period, cash-flow, cash requirements, repayments of existing loans, cash shortfall, cash flow FA-Leverage interest coverage, insolvency, debt ratio, equity ratio, debt-equity ratio, debt to equity, fixed charge coverage, collateral to debt, acquire to pay, acquire to repay, long-term solvency, long-term liabilities, current liabilities, noncurrent liabilities, short-term liabilities, cash payments, book debt, book value to equity, market value to equity, book value to capital, market value to capital, total debt, net gearing, leverage ratio, capitalization ratio, interest coverage, cash flow to debt, indebtedness, total liabilities The overall frequency of words in a given category only shows how much attention the firm's management devoted to this category in its communication with stakeholders.In the next section the hypothesis will be verified that the better results the firm in the category reached, the more space is devoted to this category in the associated text of the annual report.To demonstrate the benefits of topic analysis in the text of the annual reports, the results will also be compared with commonly used dictionaries proposed in previous studies not only for the general analysis of the text but also specifically for the analysis of financial texts.Dependencies between the dictionaries of financial analysis and other tested dictionaries will also be showed.
The set of financial ratios presented in the previous section was collected for the year 2013 from the financial statements of the annual reports of 1380 U.S. firms listed on the New York Stock Exchange (NYSE) or Nasdaq (source: http://www.reuters.com/finance/global-market-data).Specifically, financial ratios from the following categories were included: (1) profitability (ROA, ROE, and ROS), ( 2) activity (S/TA), ( 3) market (P/E, PBV, payout ratio, and dividend yield), ( 4) liquidity (cash/TA, and NWC/TA), (5) cash flow (FCFF, FCFE, and RetEar/TA), and (6) leverage ratios (BD/TA, and MVEq/BVdebt).For basic descriptive statistics, see Table 2. Legend: ROA is return on assets, ROE is return on equity, ROS is return on sales, S are sales, TA are total assets, P/E is price to equity, PBV is market price to book value, NWC is net working capital, FCFF is free cash flow to the firm, FCFE is free cash flow to the equity, RetEar are retained earnings, BD is book debt, MVEq is market value of equity, BVdebt is book value of debt.
In addition to the financial statements, the textual communication of management with stakeholders in annual reports (Item 7. Management's Discussion and Analysis (MD&A) of Financial Condition and Results of Operations in 10-K filings) was also pre-processed.The MD&A section is considered the most important part of 10-K filing in previous literature (Kearney and Liu, 2014).
To compare the performance of the dictionaries proposed in this study specifically for financial analysis, other dictionaries were also used that were predominantly employed in previous studies.The general dictionaries were represented by five general semantic features of Diction 7.0 (Hart, 2001), namely certainty, optimism, realism, activity, and commonality.These features were calculated from the series of 35 word categories.Diction 7.0 and more related resources can be obtained at http://www.dictionsoftware.com/.
However, general dictionaries performed relatively poorly in previous related studies (Loughran & McDonald, 2011;Henry & Leone, 2016).Therefore, several finance-specific dictionaries have been developed to address specific needs of financial texts.A set of extensive dictionaries were developed by Loughran and McDonald (2011).These dictionaries have received much interest in recent related literature (Hajek et al., 2014;Myskova & Hajek, 2016;Henry & Leone, 2016).The following categories are included in these finance-specific dictionaries: positive, negative, uncertainty, litigious, and modal.Word lists for each of these dictionaries can be downloaded at http://www3.nd.edu/~mcdonald/Word_Lists.html.To address the ambiguous use of positive words, a collocation analysis was performed of negation words (no, not, none, neither, never, and nobody) occurring within three words preceding a positive word.Additionally, the net positive category was calculated as (Positive -Negative) / (Positive + Negative) (Henry, 2008).In agreement with previous studies (Engelberg et al., 2012;Garcia, 2013), the raw term frequency of word categories was divided by the length of the MD&A.Thus, the length of documents was considered and all words in each category was treated as synonyms.The descriptive statistics of the dictionaries from prior studies are presented in Table 3. Similarly to the previously introduced general and finance-specific dictionaries, the raw term frequency of word categories for financial analysis were normalized to the length of documents.Descriptive statistics of the dictionaries are presented in Table 4. On average, words related to leverage ratios were present most frequently, followed by activity and market ratios.

EMPIRICAL RESULTS AND DISCUSSION
The analysis of relationships between financial ratios and dictionary frequencies was performed using Spearman correlation coefficients.First, the financial ratios were compared with the finance-specific dictionaries proposed by Loughran and McDonald (2011).Table 5 shows that positive (and net positive)

Renáta Myšková, Petr Hájek
Comprehensive assessment of firm financial performance using financial ratios and …

103
word categories were strongly correlated with all categories of financial ratios, except liquidity ratios.Specifically, profitability and activity ratios were positively correlated with positive words.Moreover, profitability ratios were also negatively correlated with negative and modal words.In terms of assessing financial indicators, therefore, this dictionary is particularly suited to looking for positive or negative comments, not only with respect to the results presented in Table 5 but also due to the occurrence of the monitored words (see Table 3).In the second run of experiments, the correlations between financial ratios and the frequencies of general Diction 7.0 word lists were calculated.Table 6 shows that activity ratios were strongly correlated with active words, and liquidity ratios were positively correlated with optimism in MD&A communication.Furthermore, a higher frequency of realistic words was positively associated with higher leverage.The negative correlations between realism and profitability ratios also suggest that realism cannot be simply considered positive in the text of managerial communication.Therefore, if the dictionary is used, comments on profitability cannot be considered very objective, but it is possible to obtain additional information about the cash-flow management activities and the use of external resources.
Thirdly, the correlations between the financial ratios and the proposed dictionaries for financial analysis were examined.Table 7 presents the Spearman correlation coefficients for these word lists.Most strikingly, profitability, activity, liquidity and leverage dictionaries were positively correlated with the corresponding financial ratios.These results correspond to our hypothesis.
However, market ratios were strongly correlated with the word list for activity, instead of market dictionary.The reason could be that the comments focused on justifying the level of market indicators in terms of asset management, respectively in terms of turnover or tied-up capital, as the majority shareholders are particularly interested in this area.The development of market indicators responds to investors' expectations.Managers are aware of this, which may be the reason why there was no such interdependence between market indicators and the market dictionary in the monitored annual reports.Moreover, cash flow dictionary was positively correlated with the liquidity ratios, rather than cash flow ratios.This may be due to the fact that comments mainly focus on tracking the ratio of short-term assets to total assets and ignoring the discussion of cash-flow itself.The strong relationship between liquidity and cash-flow dictionaries is also evident when comparing the positive and negative correlations with the liquidity and cash-flow indicators.

Renáta Myšková, Petr Hájek
Comprehensive assessment of firm financial performance using financial ratios and …

105
The final set of analysis was aimed to examine the relationships among all dictionaries.Table 8 demonstrates that the word list for activity ratios strongly corresponds to positive and activity dictionaries, in particular.The liquidity word list corresponds strongly with the Diction 7.0 dictionary.Moreover, the word list for cash flow is positively correlated with litigious and modal words.Finally, profitability word list is strongly related to the net positive tone of text.

CONCLUSION
The performed analyses showed that the firm management generally gives supplementary information justifying the financial results in the annual reports.However, the scope and structure of such information is different for well/poorly performing firms.Generally, the obtained results conform to the expectations of positive/negative sentiment categories (Davis et al, 2012;Hajek et al., 2014;Myskova & Hajek, 2016).The results of the present study suggest that firms that have good financial results verbally describe not only the achieved values of financial indicators, but they also elaborate the causes of the result in more detail.This seems to be true for both positive and negative impacts.Conversely, firms with worse economic results are concise in their comments and focus mainly on the analysis of the causes negatively affecting their economic activity.For the purpose of this research, several dictionaries were applied, including general, finance-specific and dictionaries designed for financial analysis.The results show that it is not possible to determine one dictionary that would provide the best information ability across all categories of financial indicators.On the contrary, the results suggest the following conclusions: (1) for the indicators of profitability, activity and cash flow, positive (net positive) dictionary of Loughran and McDonald (2011) is the best approach; (2) for liquidity ratios, it is the dictionary focused topically on cash flow; (3) for the leverage ratios, the dictionary focused on leverage terms performed best; and (4) for market indicators, it is not possible to clearly recommend one dictionary, it is rather preferable to use a combination of dictionaries according to the interest of investors.The results also show that the dictionaries proposed for the financial analysis are correlated with previously used dictionaries to certain degree.Their contribution is likely to be seen especially in combination with other linguistic categories, namely positive and net positive sentiment.
The authors are aware of certain limitations of the research related to both the selection of a set of financial indicators (financial ratios were preferred) and linguistic assessment in annual reports (which depends on the willingness of management to provide additional information and on the management approach to the monitored firms).For further research, it would be interesting to perform statistical analysis of individual words included in the dictionaries because the words may not be of equal importance.Further research should therefore focus on assigning weights to individual words.In our opinion, contextual sentiment (tone) assessment of financial analysis' words in the text would also be an interesting area for future research.Only the positive / negative tone of topic occurrence allows an accurate assessment of their correlations with the financial results of the firms.

Table 2
Descriptive statistics of financial ratios

Table 3
Descriptive statistics of dictionaries

Table 4
Descriptive statistics of dictionaries for financial analysis

Table 5
Spearman correlation coefficients between financial ratios and finance-specific dictionaries

Table 6
Spearman correlation coefficients between financial ratios and general Diction 7.0 dictionaries

Table 7
Spearman correlation coefficients between financial ratios and dictionaries for financial analysis

Table 8
Spearman correlation coefficients between dictionaries for financial analysis and other dictionaries