What is self-supervised learning doing when there's no labeled data?
Instead of labels, the model is trained on prediction tasks it can generate from the raw sequence itself: Next Token Prediction (predict the next transaction given the history) and Masked Language Modeling (guess transactions that were hidden from the sequence). Both objectives force the model to learn real structure in spending behavior — recurring payments, spending cycles — without any human labeling a single example.
Answered in
Why Fintechs Are Replacing Hand-Built Features With Foundation ModelsManually engineered features don't transfer between fraud, credit, and recommendation models. A transformer trained on raw transactions can.
Read the full analysisOther questions this article answers
- What's wrong with manually engineered features for financial ML models?
- How do you turn a bank transaction into something a transformer can process?
- Why not just assign each unique transaction its own ID, like a recommendation system does with products?
- What's the practical difference between late fusion and joint fusion when combining embeddings with tabular data?
More how ai actually works questions
- How many new researchers can receive the next OpenAI grant?
- When do applications open?
- Does the classifier make a final decision about an image?
- Can any developer call the classifier?
- How can Cursor answer questions about an entire codebase without storing the code?
- If embeddings are stored instead of source code, can the original code be reconstructed from them?
- Why does Cursor use a Merkle tree to keep the index synchronized instead of just re-scanning changed files?
- What happens when the server needs the actual source code, not just its embedding?
Every answer on Crashtech is written by the editor of the article it comes from — never auto-summarised. Browse all answers or the How AI Actually Works beat.