In his book Principles, Bridgewater founder Ray Dalio said: “Investing is an iterative process where you make a bet, you fail (sometimes painfully), learn something new and try again. You can improve your decision-making through trial and error.”

In early 2017, CITIC Securities established an artificial intelligence team to provide its internal institutions with a unified cloud platform for artificial intelligence application R&D, deployment, and operation, as well as a unified data, computing power and R&D operating environment.

Three years later, the team Song Qunli, Xu Changze, Hu Zhenning, and Zhang Junling jointly wrote the paper "CITIC Securities Intelligent Cloud Platform and Intelligent Application" (hereinafter referred to as "Intelligent Cloud Paper"), citing the above passage and sharing their team's framework in , investment advisory, algorithm, consulting, risk identification, document extraction, investment research automatic reporting and other 9 major applications.

"Artificial intelligence and quantification are both highly confidential jobs. Why are you willing to share these applications?" The white paper research team asked Xu Changze and Hu Zhenning.

"These applications are no longer worthy of secrecy. Everyone is going in these directions, and we are also upgrading." After a year, the market is changing, and the team's research continues to sail in depth.

01 One engine of unified cloud platform supports 9 major applications

"Our IT department is involved in the procurement work, and we know that all business departments of the company are buying computing power, but they are all fragmented and cannot be combined, which is obviously very uneconomical. Therefore, we want to create a unified platform with greater computing power and overall maintenance costs. Lower." In 2019, CITIC Securities tried to apply the latest technology to its business and lead the business with technology.

The "Intelligent Cloud Paper" clearly describes the construction of its platform. Its core function is to build corresponding services and modules for resources, integrate and maintain them to meet the needs of customers to develop and run artificial intelligence applications, including:

Financial data service: connect all internal market sources, historical market data and historical macro data sources, and provide factor research tools and custom factor data regular maintenance and sharing services.

R&D platform: supports the operation of various machine learning algorithms and various deep learning models and frameworks (LSTM model, Tensorflow+Keras/PyTorch and other frameworks), and provides development tools and model code generation tools.

Multi-tenant computing power platform: A cloud service scheduling system that shares multi-GPU computing resources and supports multi-tenancy. Supports multi-machine and multi-card parallel training, and provides hyperparameter search function.

Strategy backtesting: Supports historical data backtesting and real-time simulated trading of the trained model (completely simulates the trading scenarios of CITIC Securities, anti-knock, simulated matching and slippage).

Performance analysis: perform back-testing and real-time trading performance evaluation and risk measurement on the trained model, and provide calculation and analysis reports for performance evaluation benchmarks and risk measurement indicators.

Trading interface and model inference execution framework: Provides an API for platform training, backtesting, and optimized strategies to be packaged and submitted to the production environment and connected to the trading system.

The R&D platform also provides online modular SaaS development tools and PyCharmIDE-based clients for professional developers. It can help users lower the threshold for using and entering the field of machine learning quantification, while meeting the needs of professional developers for large-scale engineering development:

Modular development platform : a visual machine learning and deep learning model building and development tool platform, which is convenient to quickly build quantitative applications of machine learning.

Expression engine : Users may have accumulated a lot of indicators and formulas before, but if the development environment is migrated to the tool platform of machine learning, it will be difficult to develop all indicators and factors from scratch from the basic data. The expression engine provides a general solution, and users no longer need to implement cumbersome programming from scratch.

Multi-client usage mode : It is convenient for professional users with different needs to go from development/testing environment to production environment and corresponding off-site production disaster recovery environment.

The operating environment of the intelligent cloud platform provides a high-power 8GPU operating environment for the demands of development/testing users. Models developed, tested and trained by users can be directly deployed to the production environment using low-power GPU servers. Safe and efficient multi-tenant isolated production operating environment. Finally, the system also provides remote disaster recovery and regular data synchronization services for the production environment.

After the platform gradually collects the data, the platform will soon become effective. "We have intensively used the computing resources of the entire large cluster, which has rapidly improved the efficiency. The risk control department's risk analysis simulation computing performance is 400 times different before and after, and the results can be obtained as fast as a few tenths of a second." Hu Zhenning introduced: Now the team is continuing to dig deeper and empower various departments, including the expansion of target types and the expansion of strategic research. "We see the market is getting more and more coiled, excess returns are declining very quickly, and if you don't work hard, returns will get lower and lower, and volatility will increase."

02 The application of deep reinforcement learning in stock selection and trading algorithms

"Deep learning has strong perception ability, but lacks decision-making ability; while reinforcement learning has decision-making ability and cannot deal with perception problems. Therefore, combining the two machine learning methods complements each other's advantages and provides a solution for perceptual decision-making problems in complex systems. Solutions."

In the "Intelligent Cloud Paper", the CITIC AI team talked about the application of reinforcement learning. The fully intelligent algorithmic trading strategy AITWAP3 developed by CITIC AI aims to use deep reinforcement learning theory to train machine learning by learning historical A-share Tick market data. Algorithmic trading strategies (selection of timing, volume and price type). All order decisions are made independently by the machine, and the machine will continuously learn and adjust strategies according to the current market environment every day to adapt to the new market environment.

From the real launch on December 23, 2019 to May 27, 2020, the strategy has accumulated 9,134 transactions, with a total transaction value of about 1.32 billion yuan, and the average performance is 2.54% better than the market VWAP. (2.54 BP). From January 2, 2019 to November 15, 2019, the average performance of the A-share market-wide stock backtest was better than the market VWAP by 1.79% (1.79 BP).

At the same time, the CITIC team applied reinforcement learning to quantitative stock selection, but did not use it as the focus of the strategy, because in the team's view, the core anchor point of quantification should be performance as the goal, and various quantitative techniques and parameters may change , it should evaluate its own resources for exploration. When the entire market is using similar methods and becomes very crowded, it becomes a zero-sum game, Alpha will also become Beta, and in turn some simple linear or nonlinear models Can also produce good results.

In addition, in the eyes of Hu Zhenning, the current reinforcement learning is essentially a method of generating data or enumeration using Monte Carlo. If the market sets the same objective function, it will also form a very crowded strategy. . "It's counter-intuitive. You can't solve problems with advanced weapons or advanced algorithms. Prior knowledge is more decisive. Quantitative competition is the accumulation of data and the ability to iteratively and rapidly upgrade models. It's not how advanced the model is. If the factor itself is not good, it is impossible to have an advanced model, and it suddenly magically improves

Reinforcement learning is that the agent learns by trial and error, and the reward and punishment value obtained by interacting with the environment guides the behavior. The goal is to make the agent obtain the maximum reward and punishment value. However, in a large market environment, it is difficult for individuals to influence the entire market environment. How can reinforcement learning play a role?

"This environment can be the market, or it can be defined as one's own positions, so that the traditional simple strategy can be transformed into position allocation, which is more refined." Hu Zhenning explained.

03 Does AI quantification need interpretability?

Various types of data show that factor mining in the quantitative market is going in different directions. Some institutions have conducted subtle research and become more and more meticulous, while others have used machine learning and genetic algorithms to brute force the market.

Explainability is a huge problem for brute force mining. Some people think that artificial intelligence only needs to be used, and there is no need to know why, just like ordinary people spend their whole lives, and it is difficult to understand the motivation of each move of AlphaGo.

"AlphaGo is a closed game with perfect rules and no noise during operation. But there are many people, a lot of noise, and weak rules in the investment market. Even if we can use deep learning to find a theory similar to the theory of relativity, the investment opportunities There will be changes in regulation, trading rules, trading venues, market participants, and media public opinion. For example, the price limit system changes from 10% to 20%, and the amplitude and volatility increase. The original model may fail, so we cannot Use static models to adapt to changing markets.”

Xu Changze believes that the use of human mining factors, although the process of sorting out is very slow, it has very strong accuracy and interpretability, and public funds are used more. The essence of machine mining is a mathematical formula, which is effective but weak in interpretability. At the same time, there are marginal, easy and always difficult: from x+y to (x+y)2+z data and symbols are getting deeper and deeper. During the earnings period, everyone was fine, but when there was a pullback, institutions were unable to explain to investors the reason and sustainability of the pullback.

Due to the problems of data and statistical methods, the problem of mining the validity of factors has been troubled by the industry at home and abroad. How to exclude the lucky ones that rely on Data Mining (Data Mining) among the endless factors, and find the ones that can really explain the expected return of the stock The interface difference, what can really beat the market, is the focus of the industry.

This question is like a hypothesis put forward by Buffett in 1984: if 300 million people are allowed to play 20 coin tosses at the same time, after 20 rounds, there will still be about 250 people who are all right. Then these 250 people are some kind of characteristics of themselves, and they are also Just relying on luck (Lucky)?

Hu Zhenning is also carrying out a lot of work on the interpretability of auxiliary machine learning models while "brute force cracking". This type of technology is relatively mature and is mainly used in the following aspects:

1. The machine learning model interpretability technology can be well screened, filtered, and calibrated to eliminate the Lucky problem caused by DataMining in genetic algorithm factor mining and over-policy mining.

2. Quickly locate the factors that lead to product withdrawal, and under what special scenarios and conditions will the factors recur, helping to further improve models and strategies.

3. Artificial intelligence can be regarded as a mathematical mapping relationship obtained through data mining. It is necessary to pay attention to whether there is noise in the data, whether it is well corrected during training, whether there is overfitting, and whether it is filtered during generalization. The influence of noise is removed... The probability estimates are clearly judged through interpretability, whether it is caused by noise or indeed the mapping relationship learned from the data.

4. Is there a certain boundary in the mapping relationship learned by the model from the data? Has the current reality crossed the border? Interpretability can judge abnormal situations and help investors to be more aware when using strategic models.

5. The state of the whole market is not simply the three states of ups and downs, but a complex structure. Interpretability research will also study the different evolution and exposure of factors in various market states. From the past history, The market state is often in a state where several states are interlaced similar to the Gaussian mixture mode. The entry or exit of funds in the market will have inertia. When the current market state is dying, a new market state has been born, and this can be recognized. When When the state crosses a certain probability critical point, the market may undergo a drastic reversal. This process does not require quantitative prediction, but identification, judgment and confirmation in order to switch strategies and adjust corresponding positions and weights to increase returns.

04 Quantification VS Quantification Quantification VS The Power of Active Gaming

In the investment market, quantification and quantification institutions, quantification and active investment, domestic quantification and overseas quantification are exchanging chips and constantly playing games.

On September 6, Yi Huiman, Chairman of the China Securities Regulatory Commission, delivered a speech at the 60th World Federation of Exchanges General Meeting and Annual Meeting: "In mature markets, quantitative trading and high-frequency trading are more common, and they are increasing market liquidity and pricing. At the same time, it is easy to cause problems such as transaction convergence, increased volatility, and violation of market fairness.”

How to correctly view the role of quantification in the market? How are quants affecting this market?

"The contribution of high-frequency trading and quantitative trading is actually similar to cracking down on counterfeiting, making the market's unreasonable pricing more reasonable, and this force should be as large as possible. For example, when the valuation of new energy vehicles rises to 50 or 60 times, it should be With such a game force back to 45, the pricing of the entire market will be relatively stable, and the stock price will fluctuate with low volatility most of the time with performance, which is an efficient market.”

Xu Changze believes that the entire A-share market system has been designed with great emphasis on protecting the interests of small and medium investors, such as the T+1 system and restrictions on high-frequency trading. The fact that quantitative institutions continue to move towards medium and low frequency is actually a response, and the essence of daily frequency trading is not much different from active management. It can only be said that investment concepts and investment methods are more rational.

Hu Zhenning tried to understand the role of quantification on the market from another angle: "Some people have concluded that the market is divided into three types of traders: noise traders, trend traders, and institutional traders. Institutions can play a certain role in promoting the market. They are smart traders or Beta investors. Among them, the most important profit from quantitative trading is Alpha, followed by "harvesting" noise trading. Of course, quantitative institutions will also harvest each other. When machine learning is introduced into quantitative trading, it can adapt to the market. , the competition among institutions is the ability to continuously upgrade the iterative model.”

From the experience of the United States, Xu Changze believes that the excess return of quantitative institutions may be higher than that of active management. Active public fund managers are good at doing industry research, but it is difficult to switch industries. The excess return comes from the long-term growth of the industry. The compound interest rate will be relatively high, but the volatility is also high, the timing ability is weak, and the active management ability is not emphasized. Therefore, quantification not only harvests the money of retail investors, but also harvests the money of active management: ''Iron machine, flowing research, and active management investment ideas are difficult to replicate and inherit. "

Hu Zhenning also saw that there are quantitative strategies for sniping active investment, and we can see such an example by analyzing the Ticks market. However, under the game between the two sides, such transactions are quickly reduced, because the probability of "loopholes" (defects) in quantitative strategies can (be) caught is very small after all. For those who are active traders, they also need to upgrade their weapons. CITIC Securities will also help active traders to provide order placement strategies based on intelligent algorithms. It will be much stronger than the traditional standard TWAP in terms of slippage and potential positive returns.

There will also be games between quantitative institutions, including data and algorithms.

In Xu Changze's view, competition is an inevitable development. It is an industry development path that has been experienced in many industries. After mastering the technology, it will quickly enter the flood. The excess return is actually not like Beta. Beta is a positive-sum game, but excess means fighting each other, so it is attenuated, and the higher the frequency, the faster the decay, because the lower the frequency, the more information needs to be processed. , the more information that needs to be judged, the more difficult it is to judge. This is not good at quantification, and active management has greater advantages, so it is not ruled out that quantification is close to the critical value of marginal benefit.

With the decline of performance, quantitative agencies have attacked in multiple lines, improving the diversification of strategies, and even participating in new ones. "Some products are divided into about 200 million yuan, adapting to the proportion of offline new products, and they can get 5% of the income, but this part of the income may not be available in the future. Quantification will also return to the essence of investment: most of the excess income comes from For the fundamentals, high-frequency volume and price data that has no economic meaning can easily be of little value. Everyone may indeed get a little profit, but in terms of profit divided by the total scale, it does not matter much to the rate of return. I used it." Xu Changze said. 

Of course, there will also be reverse harvesting between algorithms. "In this market, everyone is pursuing machine learning, and if they use the same or similar data to continuously optimize their strategies, there will definitely be a situation where everyone is training the model to seek the highest yield, and as a result, everyone will gather the same number of people. For a similar strategy, the market is reflexive, and the crowdedness of the strategy will lead to people not making money on these strategies. Conversely, other strategies in the market that have opportunities, (from the backtest point of view) It seems that they do not make money (historical data does not seem to have their high returns), but can become a better opportunity (the actual strategy yield is very good), because they avoid the crowded (quantitative strategy) that is particularly high." Hu Zhenning said .

Overseas institutions are also new players in the market. Hu Zhenning learned that there are already domestic institutions that provide foreign capital with factors and trading systems. In his view, the entry of foreign capital will not have a particularly large impact in the short term. On the contrary, foreign capital will need to rely on some existing domestic quantitative technologies to help them understand and understand the market, but in the long run, it should have a great impact. This increase will promote the institutionalization of A shares.

Investors need to put the process of foreign investment into China into the entire national strategy and the entire market game to observe. China concept stocks in overseas markets plummeted sharply. In the future, the game between China and the United States will lead to a major rotation of the capital market. On the one hand, China concept stocks will go to Hong Kong stocks go. On the other hand, all kinds of active and passive funds will move to China's capital market. On the third aspect, the total transaction volume of the market is getting larger and larger, and the funds and games of the participants will definitely be upgraded.

For domestic institutions to deploy overseas, the quantitative and active paths are not consistent. Regardless of whether it is active or quantitative, the excess returns in Europe and the United States are relatively small now, the beta is stronger, tracking the index is a better strategy without the need to pursue excess, and the result of more effective market pricing is that the fluctuation is small, and there is no price difference. Opportunity. Quantification needs to go to emerging markets, individual investors are in it, and unreasonable pricing is also in it, so if domestic quantitative institutions want to export investment capabilities, it is definitely not an old capitalist country, but a market similar to Vietnam and India. .