Efficient training of language models to fill in the middle
We show that an autoregressive language model can learn text embeddings after applying a simple transformation to the dataset. This simply moves a range of text from the middle of…
How to predict and reduce the risk of potential abuse of language models in disinformation campaigns
As generative language models improve, they open up new possibilities in fields as diverse as medicine, law, education, and science. But like any new technology, it’s worth considering that they…
Early consideration of the potential impact of large-scale language models on the labor market
We investigate the potential impact of the Generative Pre-Trained Transformer (GPT) model and related technologies on the U.S. labor market. Use a new rubric to evaluate occupations based on their…
Language models can explain neurons using language models.
Although most of our explanations have low scores, we believe that we can further improve our ability to create explanations using ML techniques. For example, I found that you can…
FinanceGPT in South Africa simplifies financial analysis and configures interface in local language
Financial GPT is a generative AI startup built to help businesses with their financial analysis tasks. FinanceGPT (Ipoxcap) was initially built for startups without an in-house finance team and for…
MIT’s machine learning system based on light could produce more powerful large-scale language models
by Elizabeth A. Thomson, MIT Materials Laboratory September 24, 2023 An artist’s rendition of a light-based computer system that could potentially power the power of machine learning programs like ChatGPT.…