Transformer-Based Language Models Across Domain Verticals: Architectures, Applications and Critical Assessment

A new review published on arXiv organizes the rapidly expanding landscape of transformer-based language models into a working taxonomy and surveys their real-world deployments across seven domain verticals, from healthcare to scientific work [1]. The paper, posted in June 2026, addresses a persistent problem for practitioners: the sheer pace of new model releases has made it difficult to distinguish durable architectural ideas from incremental announcements [1]. The authors construct a taxonomy that spans encoder-only, decoder-only, encoder-decoder, long-context, permutation-based, and generator-discriminator variants [1]. They then extend the discussion to post-2023 developments that changed the picture in practice, including instruction tuning, reinforcement learning from human feedback, direct preference optimisation, mixture-of-experts scaling, and retrieval augmentation [1]. The review examines flagship model families from OpenAI, Anthropic, Google, Meta, Mistral, and DeepSeek [1]. DeepSeek, a Chinese AI company founded in July 2023 by High-Flyer co-founder Liang Wenfeng, launched its R1 model in January 2025 alongside an eponymous chatbot [11]. The company claims it trained its V3 model for US$6 million, roughly one-sixteenth the reported cost of OpenAI's GPT-4 in 2023, while using approximately one-tenth the computing power consumed by Meta's comparable Llama 3.1 model [11]. DeepSeek's models are described as open-weight, with parameters openly shared but training data not openly licensed [11]. The review surveys deployments across healthcare, finance, legal, education, customer service, creative writing, and scientific work, linking each domain to the specific capabilities that make a transformer the appropriate tool [1]. The authors compare architectures on four axes that matter to deployment decisions and quantify the trade-off between parameter count and energy cost [1]. They also discuss how alignment methods, data provenance, and benchmark saturation change what it means to call a model state of the art [1]. A separate study published on arXiv in 2025 examined how LLMs from OpenAI, Google, Anthropic, DeepSeek, and xAI performed in quantitative sector-based portfolio construction [9]. That research found a strong temporal dependence in LLM portfolio performance: during stable market conditions from January to March 2025, LLM-weighted portfolios frequently outperformed sector indices on both cumulative return and risk-adjusted measures, but during the volatile April-to-June 2025 period many LLM portfolios underperformed [9]. The findings suggested that current models may struggle to adapt to regime shifts or high-volatility environments underrepresented in their training data [9]. Another 2025 arXiv paper tested nine model configurations from four providers on 90 multi-turn scenarios designed to surface misaligned behavior in conflict contexts, including false equivalence between documented atrocities and denial of genocide [6]. Failure rates spanned 6 percent to 47 percent between the best- and worst-performing models, and when users pushed for balance in cases where international courts had already assigned responsibility, five of nine configurations failed 80 to 100 percent of the time [6]. The review's final section lists research questions the authors believe deserve more attention [1].