Frequently Asked Questions

 Founded in March of 2023, PerformLM optimizes language model training and inference by reimagining the structure, architecture, and training process from a first principals perspective. Using this multi-pronged approach, PerformLM’s optimization achieves performance that is comparable to best-in-class models which are trained on curated datasets, all while using exclusively open-source data. Our optimization helps models run over twice as fast as similarly-sized competing models. 

PerformLM leverages both improvements in training as well as architecture to increase both model performance and speed.

Our training improvements center around a combination of a specialized curriculum learning methodology and dynamic label smoothing technique. This union of optimization techniques allows our models to learn over “messier” open-source data as though it were a curated high quality dataset (e.g. as used in Microsoft's "Textbooks Are All You Need"), allowing for the performance from expensive smaller datasets to scale to open datasets with trillions of tokens.

Our structural improvements focus on both simplifying model block computations in accordance with the results of layer-wise ablation analysis, as well as improving CUDA kernel fuse-ability. These improvements result in over a 2x speedup over a comparable sized standard GPT-style or Llama-style model.