Hey guys, welcome to a brand new module.
In this module we're going to be learning all about Data Persistence,
and that means databases.
In the last module when you built your own API, you might have noticed a peculiar thing.
You might have created some new blog posts,
right?
And as soon as you restarted your server, if you stored any of those blog posts in your variable,
then you'll notice that those blog posts completely get wiped and returned to their original state.
Why is that?
Well, that's because your data is held in a temporary form of memory.
It's only valid as long as your server state doesn't get refreshed.
So how can we persist our data?
Well, then we would need some form of data storage and not something like this.
This is an actual memory disk that I found in the Computer Museum in Bletchley Park, where they had
this gigantic thing,
you can see how big it is relative to my face.
And believe it or not, this entire thing only stores four megabytes.
It's crazy how far we have come in terms of technology, but under the hood, these things still work
more or less the same.
We have ways of putting data into a permanent state of memory.
So in these cases, which are magnetic based memory storage, what you have is an electromagnet that
can generate a polarity in this electromagnet.
And as you have a strip of little magnets moving around, by changing the polarity, by changing the
current that's going through the electromagnet, we can flip these poles around so we can make this
part North, and then this part South, and then this part South again, and this part maybe North,
and essentially what happens is when you read this data and you look at all the little magnets and you
look at the polarity of it, the computer is able to tell what the bits that are stored on this.
For example, if you have two magnets that are of opposite directions, so this, North/South are flipped
from each other, then this would be a 1.
And if they are the same, it doesn't matter which direction they are counted as a 0.
And you can basically go through this entire strip of memory and store data in bits and bytes by changing
the magnetic polarity.
So this is actually what I learned at this museum.
If you're ever in London and you have a minute to go up North on a train, I highly recommend going
to the Computer Museum in Bletchley Park.
But going back to data storage, what we need essentially is a form of data storage that prevents our
variables, which are held in temporary memory, from going poof.
And that means we have to talk about Databases.
Now there's a whole bunch of different types of databases, but the main types that you'll hear about
is SQL Database and a NoSQL type Database.
SQL stands for Structured
Query Language.
And if you will, it's sort of like a programming language that allows you to query a database that
is in this particular structured format.
So we're already getting close to discovering the differences between these two types of databases,
structured and not structured.
The NoSQL stands for well, no SQL,
but also some people call it not only SQL but let's take a closer look at what makes each of these
types of databases special.
Let's start with SQL database.
This is the OG, the Big Daddy, the database that has been around for years and years, ever since we
started needing to organize data,
these types of databases have been around. And the big principle of SQL databases is that structurally,
data is stored in tables.
For example, you have a website and you need to keep track of your users,
so you build a users table and beforehand you have to anticipate what do you need to store in this table.
So you probably need to store their name, their password, their email.
All of these fields that you start creating determine the structure and size of this table.
Once you've created all of the columns, then you can populate it line by line by adding new records.
So you sign up a new user, you add a new record and you store their data.
One of the greatest things about SQL type databases is their ability to form relationships,
and this is why you'll hear another name for this type of database is a Relational Database.
And this is because let's say we have a user here,
let's say it's me and I have a password, and I have an email and I create a blog post.
So then in this case, we've already created the record in the users table of this user, me,
and now we're creating a new blog post.
So in this blog post we have a title, we have a date of creation, and we have an author.
And because this is an SQL database, it's incredibly easy to link up the relationship between that
field and this record.
So now we have a relationship and instead of creating one very long line of record, we can now have
multiple records on another table that's linked to one user.
So then you imagine I write a second post and you can link this also to me as a user.
So then I can query all of the posts that I've written very easily by using the Structured Query Language.
Some of the most popular types of SQL databases that you might have heard of is the Oracle Database,
which is currently worldwide, one of the most used databases in large corporate organizations,
but they do cost something like $50,000 as a starting plan.
So we're not going to be looking into that until we're like a fortune 500 company or something.
But there's also other ones like Postgres, MySQL, SQLite, et cetera.
And there's a lot of different brands of databases out there.
The difference is that some of these are open source, like Postgres, like SQLite, and some of them
are not open source like Oracle or Microsoft SQL.
So that's where the free and paid part comes in.
That's a good thing to be aware of when you're doing your research.
The other type of database that we mentioned are the NoSQL type databases.
And these are characterized by one major advantage, it's their flexibility.
For example, if I created a document in almost like a JSON format, which you're pretty familiar with
by now. So I'm creating my new user,
it has a name field, an email field, a password field,
and just like with JSON, and just like with JavaScript objects, one of the greatest things about NoSQL
databases is you're able to change the structure of your data afterwards without having to change the
entire database.
So this means that let's say I have a user here, and I decide that for this user and for this user
only, I am going to give them an extra field.
So for Angela, and only for Angela, the user is going to have an extra field of favorite food and I'm
going to store it as, well,
my favorite food is probably noodles, but I do love all the carbs.
So.
But anyways, this means that even if we had other users with a particular name, an email, a password,
and no field of favorite food, so we didn't plan to have this particular field,
we can modify our data on the fly, and this is what gives NoSQL databases their flexibility.
They're not obliged to hold to the original structure of the table that was created at the time when
you decided to build a users table.
And you can see that this comes with a lot of advantages.
It means that your database is more flexible.
It means that as your product grows and you think of new things that you want to add to your storage,
you can do that very flexibly.
And NoSQL databases were really created in many ways to address some of the pains that people felt using
SQL type databases.
Firstly, the SQL language is probably not the most intuitive and requires learning, whereas NoSQL
uses things such as key/value pairs like what you see here, or document models where you store everything
in a single document, just like a JSON.
And it kind of meant to make things a little bit easier.
The other big idea, as I mentioned, is flexibility and moving away from this strict schema.
So having to anticipate and plan your data ahead of time.
And the other thing that they really wanted to address is scalability.
So for your data to be able to scale horizontally and vertically. So vertically means having more records,
and horizontally means having more fields.
So just as you saw now, it's very easy to add another field to a database that's built on NoSQL just
as easily as it is to add a field in a JSON.
Some of the most popular NoSQL databases that you might come across include MongoDB, Redis, or Redis,
I'm never really sure how to pronounce this one,
and DynamoDB.
Some of these are open source, such as Redis, and others are run by companies, for example MongoDB
and DynamoDB.
DynamoDB is actually run by Amazon. In terms of free, and paid, open source,
closed source,
there are also differences in this category as well.
If you've ever seen that ad of the PC guy and the Mac guy, then over the last few years, especially
2010 to 2018, I would say the general consensus has been that SQL is a little bit more traditional,
he's the PC guy, he's old school, and the NoSQL guy is cool,
he's hip, he knows how to make coffee really well, and he's just great
and he's the new kid on the block.
And the reason why people were so excited about NoSQL is because SQL has been around for ages, and it
really was this new innovation on something that's a little bit dusty databases.
Let's be fair, it's not the most exciting topic.
But as time has gone on, and as somebody who's been in this space for a long time, I'm really seeing
the tides change for SQL and NoSQL.
There's a lot of blog posts on Hacker News about how NoSQL is not as great as we initially thought it
was going to be, and a lot of people who used it for large infrastructure projects started having issues
with precisely that thing that it was going to address scalability, but also with consistency.
And it turns out that, hey, you know, boring guy is really good at relationships.
Being able to join tables, being able to query efficiently actually vastly increases the speed of database
access.
In addition, having that structure, being able to plan your database, plan your relationships ahead
of time, and having a consistency that you can rely on, turns out, is really good for maintaining
good code quality.
So it means that you're not querying for something and then suddenly get back something you don't expect
because, "Oh, we were able to dynamically add fields and change the structure."
So it's almost a bit like that meme.
I don't know if you've seen where teenagers, when they watch Lord of the Rings, they really like Legolas,
and then when you turn into your 20s, you start liking Aragorn because he's cool and he's hip, and
then actually who you really want is Sam because he's consistent and reliable. In SQL and NoSQL database
debate, which is ongoing, by the way, people are still shouting at each other, people in different
camps saying mine is better than yours.
ET cetera.
But if we look at some hard stats, looking at the StackOverflow surveys, you can see that in terms
of database technology, the clear winner is Postgres, which is an open source SQL type database.
This is true across the board.
So it's true for all respondents to their survey.
And they do these big surveys across all programmers.
But most importantly, if you look at professional developers, this is what really matters for us.
We're trying to become professional tools in web development, and it's important to learn the tools
that modern developers prefer to use.
And when you look at learning to code, you can see that it slips down a little bit because MySQL is
a lot simpler, SQLite is even simpler, and MongoDB, which is the only NoSQL database, has made
it onto second place in this list.
And if you took the previous version of this course, you'll know that MongoDB was in fact the technology
that I taught in this Web Development course as well,
but over time, with the change in technology and with the change of tides, it's important for me to
update you on what is the most relevant and the most up to date technology to learn for professionals.
That is no doubt going to be Postgres.
So that's what we're going to be learning in this course.
It is free to use,
it's used by professionals,
it's massively scalable,
it's super reliable,
and it has all of the great aspects of a good database for us to be able to use as web developers.
So in the next module, we're going to be diving deep already in learning the SQL language.
And we're going to learn how to insert how to do CRUD operations using SQL and how to establish basic
relationships.
So for all of that and more, I'll see you on the next lesson.