How do you turn a bank transaction into something a transformer can process?
A transaction has structured fields (amount, date) and free text (merchant name) — neither a pure-text nor a pure-structured encoding handles both well. A hybrid scheme tokenizes each field separately: a categorical token for the amount's sign, a bucketed token for its magnitude, separate date tokens, and subword tokens (like Byte Pair Encoding) for the merchant description — producing a compact sequence that preserves structure without ballooning token count.
Answered in
Why Fintechs Are Replacing Hand-Built Features With Foundation ModelsManually engineered features don't transfer between fraud, credit, and recommendation models. A transformer trained on raw transactions can.
Read the full analysisOther questions this article answers
- What's wrong with manually engineered features for financial ML models?
- Why not just assign each unique transaction its own ID, like a recommendation system does with products?
- What is self-supervised learning doing when there's no labeled data?
- What's the practical difference between late fusion and joint fusion when combining embeddings with tabular data?
More how ai actually works questions
- How many new researchers can receive the next OpenAI grant?
- When do applications open?
- Does the classifier make a final decision about an image?
- Can any developer call the classifier?
- How can Cursor answer questions about an entire codebase without storing the code?
- If embeddings are stored instead of source code, can the original code be reconstructed from them?
- Why does Cursor use a Merkle tree to keep the index synchronized instead of just re-scanning changed files?
- What happens when the server needs the actual source code, not just its embedding?
Every answer on Crashtech is written by the editor of the article it comes from — never auto-summarised. Browse all answers or the How AI Actually Works beat.