Forecasting cryptocurrency prices has been a challenge due to their high volatility, non-stationary and regime-dependent dynamics. Recent advances in deep learning, particular Transformer architectures, have shown their superiority in time series forecasting by leveraging self-attention to capture long range dependencies. However, studies have shown that the canonical Transformer model does not consistently outperform recurrent architectures such as LSTMs when applied to cryptocurrencies. This thesis investigates the limitations of the canonical Transformer architecture for cryptocurrency forecasting and systematically examines a series of architectural enhancements, including different patching strategies, channel-dependent versus channel-independent processing, and retrieval-augmented mechanisms for generating informative contextual inputs, aimed at improving the model’s predictive performance on forecasting future cryptocurrency prices. Building upon the insights from the empirical experiment results, this thesis proposes a Transformer-based predictive framework, MSPRACFormer (Multi-Scale Patching with Retrieval- Augmented Context Transformer). The framework supports both single-scale and multi-scale patching, and incorporates a RAC module for contextual retrieval. Each module is designed as modular components that can be flexibly incorporated into existing forecasting architectures. The framework allows dynamic configuration depending on the dataset characteristics and model architecture, thereby improving flexibility and generalization in time series forecasting. Empirical evaluations on forecasting future cryptocurrency prices using historical data from 1 January 2020 to 1 January 2025 of the top-10 cryptocurrencies confirm that this design delivers superior performance in capturing complex temporal patterns within the market.