Unveiling the Role of Computer Vision in Financial Services

Computer vision, a dynamic and ever-evolving field, has proved invaluable to the financial services sector with applications such as Optical Character Recognition (OCR), object detection and recognition, video analysis, and more. At Borealis AI, we address problems within the financial services sector, discovering that our research and the machine learning models we employ share a striking similarity to those found in computer vision.

For example, machine learning model architectures used in object recognition, activity recognition, or motion analysis/tracking – like transformers, recurrent neural networks, or other deep learning models – are also used for modeling financial data. Let’s dive deeper into the technical aspects of these methods:

• Transformers are neural network architectures designed initially for natural language processing tasks but have shown remarkable results in computer vision too. Their ability to handle sequence data, capture long-range dependencies, and parallelize operations make them excellent candidates for financial data modeling. Their key components, self-attention mechanisms, provide weightage to the significant elements in the data sequence while ignoring less relevant ones, a trait particularly beneficial for finance applications like anomaly detection in transaction data.
• Recurrent Neural Networks (RNNs), including variants like Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRUs), have been employed due to their ability to process sequential data, making them ideal for time-series analysis, a common data type in finance.
• Deep learning models, capable of learning high-level abstractions from raw data, have found their place in finance. For instance, Temporal Convolutional Neural Networks (TCNs) can analyze temporal financial data for forecasting by treating time-series data similarly to image pixels in the spatial dimension.
• Video forecasting and autonomous driving prediction have practical applications in solving financial time-series prediction problems, such as transaction prediction or stock price prediction. The models used for video forecasting analyze and learn temporal patterns, extending these patterns to the future, which is quite similar to predicting a stock’s price trajectory based on past performance. Similarly, autonomous driving prediction algorithms estimate future states based on current and past states, an approach that mirrors prediction methods in finance.
• Graph-based methods model dependency between various objects and agents can be utilized to model interactions among customers. In the financial context, these techniques enable an understanding of the interconnectedness of different financial entities and can highlight systemic risks.
• Representation learning techniques, including self-supervised learning approaches, adapt to the type of data we encounter in finance. Self-supervised learning allows models to learn representations from the data itself without explicit labels, which can be extremely advantageous when dealing with massive unlabeled financial datasets.
• Online learning, learning from new data streams, and avoiding catastrophic forgetting have widespread applicability in finance. Online learning techniques allow the model to update its parameters as and when new data comes in, which is crucial for the dynamic nature of financial markets. Similarly, preventing catastrophic forgetting is critical to maintaining the robustness of financial models as they continue to learn and adapt.
• We work with multi-modal data — integrating various sources of client interactions, such as mobile apps and transactions. This approach draws from techniques akin to those in vision+language or vision+other modalities lines of research, where multiple data sources or formats are combined to enhance the model’s performance and comprehension. Another dimension of our work involves structured domains (e.g., multiple stocks and sectors they are organized by), where we utilize techniques akin to those in scene understanding. These methods help analyze the market structure, the interconnections between various sectors, and their collective impact on individual stocks.

Research is at our core

Conducting cutting-edge research also requires a diverse range of perspectives, determination, and creative thinking. This allows us to push the boundaries of what is currently possible in machine learning. In the world of finance, we value researchers from a variety of backgrounds across artificial intelligence, including computer vision, machine learning, natural language processing, and more. Our researchers, over 40 of whom hold PhDs in computer science, physics, computational finance, mathematics, and related fields, have the resources and opportunities to undertake impactful work.

Our research team works directly with engineers, product, and business experts to transform research and prototypes into innovative products for financial services, thereby contributing to advancing the state of machine learning and shaping the future of financial services.

Real-world impact through scientific pursuit

We organize our research under three core North Star focus areas, each typically formulated as a challenging problem we are intent on solving. Here, we aim to delve deeper into the scientific complexities of these focus areas:

Asynchronous Temporal Models (ATOM)

We build machine learning models capable of making inferences from partially-observed, multi-source, asynchronous temporal data. These are the types of data commonly found in banking applications — from various types of transaction data to client interactions with our banking services.

Non-Cooperative Learning in Competing Markets (Photon)

We build models for Capital Markets data, where challenges include low signal-to-noise ratio, structured prediction, and game theoretic impacts from decisions we make.

Causal Machine Learning from Observational Data (Causmos)

Our machine learning products drive decision-making, so ultimately, we need models that can answer causal questions and generalize out of distribution.

Our commitment to Responsible AI

Borealis AI places a significant emphasis on Responsible AI, which lies at the core of our work. As artificial intelligence continues to permeate daily life, ensuring its development and use adheres to the highest standards of ethics and responsibility has become a top priority. Given the highly regulated nature of the financial industry, our commitment to Responsible AI is not just a preference but a necessity. We undertake research to tackle key issues in this domain, including adversarial robustness, explainability, and fairness, contributing to a safer, more transparent, and just application of AI.

Issues such as Explainable AI, fairness, accountability, privacy, transparency, and ethics form a core part of computer vision, and they are paramount at Borealis AI. Here, deep integrity and building responsibly form the nucleus of our work, affecting millions of people’s lives and financial well-being. The technological advances from CVPR (Conference on Computer Vision and Pattern Recognition) literature can usually be extracted for specific tasks. Still, the underlying models and optimization techniques can also be widely adopted for financial applications.

Techniques such as transfer learning, active learning, and adversarial training, commonly found in CVPR papers, can be tailored to improve financial models’ efficiency and robustness. Overall the techniques and methodologies from the world of computer vision offer valuable insights and tools to tackle the myriad of challenges within financial services. At Borealis AI, we strive to harness these possibilities to build solutions that are innovative, ethical, and make a genuine impact on people’s financial lives.

Transforming the lives of millions by advancing AI in Finance

Borealis AI, a research-focused arm of the Royal Bank of Canada (RBC), one of the world’s largest banks, is committed to creating a profound impact on millions of lives through the development and application of cutting-edge artificial intelligence and machine learning. By using our research, we aim to enhance financial well-being and provide innovative solutions to complex problems. The unique advantage we have at Borealis AI is our access to a wealth of data, intricate problem scenarios, and deep domain expertise, all courtesy of our affiliation with RBC.

Want to advance AI in finance with us?

Borealis AI is looking to hire for various roles across different teams. Visit our career page now to find the right role for you and join our team!