Skip to main content

2024 Progress...

My team has made considerable advancements in applying various emerging technologies for IMG (Investment Management Group).

Predictive Models

We have transitioned from conventional methods and refined our approach to using alternative data to more accurately predict the CPI numbers. Our initial approach has not changed by using 2 models (top-down & bottoms-up) for this prediction.   So far we have outperformed both our larger internal team and major banks and dealers in accurately predicting the inflation numbers. Overall roughly 80% accuracy with the last 3 month prediction to be right on the spot. We have also developed predictive analytics for forecasting prepayment on mortgage-backed securities and predicting macroeconomic regime shifts.


Mixed Integer Programming  / Optimization

Another area of focus is on numerical optimization to construct a comprehensive portfolio of fixed-income securities for our ETFs and Mutual Funds. This task presents numerous constraints and fundamental market structure challenges, such as the unknown prices of bonds and the availability of such bonds or the liquidity of the market. Ongoing challenges and pushing the boundaries of what’s possible in this field are both interesting and reminders of the long journey ahead for the team.


Language Models

On the language model front, we have continued to explore and improve the accuracy of the output along with solving the table extraction problem that is prominent in any LLM in 2024. The pending paper on MoA (Mixture of Agents) is ongoing to share one of the foundational points of our research and hypothesis in the last 3 years. 


Day Job vs. Night Job

Language Model continues to have a special place in my heart. My exploration into AI/ML began with reading papers on neural networks and evolutionary algorithms back in 2004. This curiosity has evolved over time and eventually culminated in my current role at Vanguard back in 2022. 

My career transitioned from being an accidental trader in 1995 to focusing on the application of AI/ML research in finance.

Following are my thoughts on May 2024:

- Need to speed up on the team's pending paper on MoA "Mixture of Agents" as more and more are thinking in this direction.

- Keep the lead on internal products my team created fiChat based on this MoA architecture.

- What matters when it comes to applied AI/ML in finance?

- Does fiChat need to feel more human to increase human engagement?

Interesting paper. It's a paper researching what exactly changes within a model when you fine-tune it concerning attention head.

[2402.14811] Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking 


Comments

Post a Comment

Popular posts from this blog

What matters?

 What matters? Six things that matter in LLM in July 2024. 1) Scale of the model, number of parameters: Scale with brute force alone won't work. But the scale does matter depending on the overall goal and the purpose of what the LLM is trying to solve.   2) Compute matters: Even more than ever, we need to look at the infrastructures around LLMs. Infrastructure is also one of the main constraints for the near term and strategically provides an advantage to a few Middle East countries. 3) Data, quality & quantity. It remains true that high-quality data with extensive (longer) training is the way. Quantity of the data also matters. 4) Loss function matters: If your loss function isn't sophisticated or incentivizes the "right" thing, you will have limited improvement. 5) Symmetry or architecture: Do you have the correct architecture around your model(s) and data? Inefficient engineering can be costly to the overall performance and output. There are inherent structural...
Post a Comment
Read more

Research Paper on MoA (Mixture of Agents)

Despite one-year setback... MoA is All You Need: Building LLM Research Team using Mixture of Agents My first attempt at using NotebookLM to create the podcast from research papers. YouTube  url on the research paper. Currently looking for ways to improve my consumption of the relevant research papers. Anyone know of a good platform that can turn the .wav files to realistic video?
Post a Comment
Read more