Can I Use SQL For Deep Learning?

You may have heard Microsoft (or Oracle, IBM, Google and even a hot dog vendor at a baseball game) touting AI or Machine Learning capabilities of SQL Server:

https://learn.microsoft.com/en-us/sql/machine-learning/what-s-new-in-sql-server-machine-learning-services

https://learn.microsoft.com/en-us/sql/machine-learning/sql-server-machine-learning-services

Of course, when it comes to the AI, data science etc., SQL Server is not the only product or service Microsoft is offering nowadays. 

This article though is about the standardized ANSI-SQL coding language. 

What role SQL language can play in Deep Learning technologies? 

Deep Learning doesn't happen without data. We need data, lots of it, to train the algorithms. And we need a place to store this huge amount of information and a way to get it into our algorithms so it can learn from it. And this is where SQL comes in. While SQL language specification was created for the database systems, especially the relational databases, you will find SQL being used in well beyond the database platforms. Even the so-called NoSQL uses SQL like language!  I think the industry standardization of SQL language has a lot to do with it's such wide and pervasive adoption.

Since deep learning requires very large amount of data, SQL is extremely valuable tool, I might say knowledge of SQL or SQL like language becomes necessary at some point. 

On the other hand, the AI tools have become so good that some of them can even can code their own SQL queries, write Python, C++ code etc. and even extend or develop new AI models just from learning from existing or new, ever evolving data!


How SQL can be used for deep learning? The short answer is, to prepare and preprocess data before feeding it into machine learning algorithms. This typically involves activities akin to the ETL (Extract, Transform &  Load), for example load the data into a model, clean and optimize (performance optimization), analyze and train the model. 

Here is a simplistic example:

Retrieve Data:

SELECT year, location, age-group, gender, income-bracket, transaction_count, total_amount FROM people

Data Cleaning:

DELETE FROM People 
WHERE year IS NULL
   OR location IS NULL
   OR income-bracket IS NULL;

Transform

SELECT year, location, age-group, gender, income-bracket, avg(total_amnount) FROM people
GROUP BY year, location, age-group, gender, income-bracket;

Data Sampling:

SELECT * FROM people WHERE RANDOM() < 0.2; 

Engineer the data:

SELECT first_name, large_name, DATE_PART('year', NOW()) - DATE_PART('year', birthdate) AS age
FROM people;

Data Splitting:

-- Split data into 70% training and 30% testing

SELECT * FROM people  WHERE RANDOM() < 0.7;
SELECT * FROM people  WHERE RANDOM() >= 0.7;


Eventually though, the actual machine learning work of modelling and training the data is often done using the specialized libraries like PyTorch (developed by Facebook), TensorFlow (developed by Google) etc., often in the Python programming language. While Python does fully support SQL, those libraries per se work with data in other formats like CSV, Pandas etc., which Python is capable of doing by retrieving the data using SQL then converting into whatever format that library supports.