Correlating human ratings of idiom transparency and decomposability with computational semantic relatedness: a cross-linguistic study of Italian and English idioms.

Idioms are traditionally defined as non-compositional multiword expressions (Cacciari & Tabossi, 2014) – the meaning of an idiom is ‘more than the sum of the meanings of its parts’. While it has been long held that idioms are fixed expressions, possibly stored as whole chunks in memory, modern research has shown that not all idioms are the same (Libben & Titone, 2008). Idiomatic expressions display different degrees of syntactic flexibility and semantic modifiability (Geeraert et al. 2017; Mancuso et al. 2020). Moreover, idioms differ in their decomposability and transparency (Geeraerts, 2002), two key variables in understanding the relationship between idiom literality and figurativity. Idiom transparency is the degree to which the figurative meaning of an idiomatic expression can be intuitively inferred from the ‘literal interpretation’ of such an expression (or how the literal meaning may have motivated the idiomatic sense; Vega Moreno, 2005). Decomposability or analyzability refers to the degree to which it is possible to observe or infer how different components within an idiomatic expression contribute to the overall figurative meaning (Tabossi et al. 2011). The common example for this is ‘spill the beans’ meaning ‘reveal secrets’, where ‘spill’ corresponds to reveal’ and ‘beans’ corresponds to ‘secrets’. Idioms are considered opaque, nondecomposable, if no such correspondence can be observed, e.g., ‘kick the bucket’ and ‘die’ (Nunberg et al. 1994).

Researchers in psycholinguistics have introduced idiom norming studies where human participants rate idiomatic expressions on different aspects, such as familiarity, ambiguity, transparency, and decomposability (Bulkes & Tanner, 2017; Titone & Connine, 1994). Such ratings have helped to document the variability of idioms on many dimensions, and in different languages, and are commonly used for psycholinguistic experiments on idiom comprehension.

In this study, we investigate how computational semantic relatedness between an idiom and its paraphrase informs our understanding of idiom transparency and decomposability. With this aim, we utilize a dataset of 150 idiomatic expressions (Pagliai, 2023) rated by native speakers in two languages (English and Italian). Specifically, for that dataset, participants provided subjective ratings for idiom decomposability and transparency, using 5 point Likert scales. Each idiom in that dataset was also paired with a literal paraphrase of its figurative meaning. In the current study, we embed the idiomatic expressions into dense embedding vectors. We also embed the paraphrases of the idiomatic meanings into embedding vectors. We then compute pairwise cosine similarity scores between the idiom embeddings and their paraphrase embeddings, estimating how semantically related an idiom (taken literally at ‘face value’) is to its figurative meaning. Finally, we calculate correlations between the obtained cosine similarity scores and human ratings of idiom decomposability and transparency. Results indicate a weak to moderate correlation (Pearson correlation values in the range of 0.3 to 0.5) between cosine similarity scores and human ratings of idiom decomposability and transparency. This indicates that semantic relatedness (or similarity) between the literal and figurative meanings of an idiomatic expression may contribute to the perception of decomposability. These findings are consistent across both English and Italian and are replicated across different embedding spaces, including multilingual models. We present the technical details of this work, discuss the assumptions involved, the tentative conclusions, and some implications for future studies.