Classical Machine Learning Notes and Basics — Statistical Arbitrage

I have been an avid learner of statistics and machine learning. The primary reason for exploring them has been the ability to apply them to use-cases cutting across different domains and problems. This included problems across auction, pricing, survival analysis and so on.

I have had the opportunity to explore healthcare, trading and sports analytics in the form of cricket. Over the years, the effort has been to be able to solve problems end to end which means, taking a problem statement, preparing the data pipelines and finally putting the model into production.

Questions: Machine Learning Basics

Gradient Descent

  • What are Convex and Non-Convex Problems?
  • How does the relationship between the Rate Of Learning and Step Size change?
  • Gradient Descent vs Stochastic Gradient Descent?

Data Modelling

  • What is Feature selection, transformation and extraction?
  • Learning vs Memoization?
  • What are the assumptions behind IID, stationarity and the same sample data?
  • Compare Train vs Validation vs Test Sets?

Feature Regularisation

  • How to do Feature Representation?
  • What is Multi-Collinearity?
  • Following, Bias Vs Variance Trade-Off?
  • Lastly, Lasso vs Ridge Regression?

Algorithms

  • Bagging vs Boosting?
  • What are Boosting Base Models?
  • What is the Logit Function?
  • Better Data vs Better Model?

Machine Learning Essentials

  • Hessian Matrix vs Gradient Descent
  • Formula description
  • What is Derivation
  • Correlation vs Covariance
  • Covariance as graph
  • What are Eigen Values Intuition
  • What is a Value Vector
  • Eigen Value as Hinge

Laplace Transformation

  • Laplace Transformation Intuition
  • Transformation as a tool

Lagrange Multipliers

  • Lagrange Multiplier Intuition
  • Constrained Optimisation Problem

Data Science with R (Scheduling using Optrees)

  • DAG
  • Shortest Path Algorithm
  • Encoding the Problem as DAG
  • Solving using Optrees

Machine Learning Regression (Interpretation)

  • Null Hypothesis
  • Significance vs Non Significance
  • Intercept
  • Interpretation

Consistently, I would often end up re-using similar problem-solving approaches. Likewise often wanted to remember all the cool tricks/hacks and hard-learned concepts I have had been able to work out all these years.

And several other contests during campus days !!

Besides improving your odds of winning these, I decided to write some notes to help collate concepts. Also takeaways from ML in an intuitive fashion. If you are one of those guys who love diagrams and moreover intuition over maths to better understand stuff, you would love them.

Machine Learning/Deep Learning For Quant Finance

Originally published at https://statarb.in.