Web Programming with Flask – Intro to Computer Science – Harvard’s CS50 (2018)

[MUSIC PLAYING] DAVID MALAN: All right. This CS50 and this is
lecture 7 wherein we pick up where we left off last time
in talking more about Python, but this time using Python for
even more powerful purposes, an alliteration I didn’t intend
until I just said it there. But the goal here with
Python now is to actually use this language to generate another
language, in particular HTML. And what we’ll do is start to provide
all the more of a mental model for what you would call
separation of concerns, whereby when you design more complicated
web based, or rather when you design more complicated software, you tend to
try to adopt certain design patterns so that you’re not just writing
everything in one big file that gets a little confusing, unwieldy
to maintain and to add to over time. But you start to separate your
concerns, and your functionality, and your features, and
your collaborators’ work into different files and
different methodologies. So we’ll try to give you some of these
industry standard approaches today and why they actually solved problems. But, of course, how did we get here? A few weeks ago, we
talked about this thing, this virtual envelope
that just represents packets of information, zeros and ones,
going across the internet from point A to point B. And among the
messages in these envelopes might be for web requests. And we talked about HTTP, a protocol,
hypertext transfer protocol, that just sends message like
this inside of the envelope. And this is requesting, of
course, the default page of a website as implied by the slash. And in many cases, but
not always, this implies a default file name of index.html. But that’s a human
convention, not a requirement. Hopefully, the server responds with
a numeric code equal to what integer? 200, hopefully, a number
that we never actually really see unless you
get more comfortable and start poking around
Chrome’s inspector and you look at your network traffic. Right. So here is an actual
successful response, which means everything is indeed OK. And if the browser looks a little
farther down in the virtual envelope, if you will, it will see another
language called HTML, HyperText Markup Language, which is the stuff
we wrote a couple of weeks ago and you’ll continue writing this week as
you now dynamically generate websites. But what we’ll introduce today is
what’s also known as a framework, or, more precisely, a microframework,
because we’ll see in just a few minutes time that it can actually be
really annoying and really tedious to generate websites
automatically using Python if you have to write Python code alone. And so humans, over
the past several years, have noticed, wow, every time I
make a web application using Python, I find myself typing
the same lines of code again and again just to get started,
or when I’m collaborating with someone else, I find that, ooh, it’s
a little hard to collaborate if we’re all working in the same file. So humans have invented
what are called frameworks. And this is an example
of code written in Python but using a framework called Flask. So whereas Bootstrap, which
you’ve played with a little bit, is a framework for CSS and JavaScript,
they have some dynamic features as well, Flask is a
framework for Python that just gives you some library
code that you can use freely in your own applications. And it just makes it, ultimately, a
little easier to get your work done and build the thing you
want to build as opposed to having to reinvent the wheel again
and again like people before you. And so here is a methodology to
which I alluded earlier just to have a mental model for the coming weeks. So up until now, pretty
much every program we’ve written in C and
most recently in Python you could call like controller code
or controller logic or business logic. Right. It’s all about getting
something done logically, usually in one file, maybe two files if
you have another helpers.ce or header file or whatnot. But we’ve been writing logic. But today and two weeks
ago when we will focus on web programming in HTML and CSS,
there’s the second part of our world. There’s like the logic that gets the
interesting work done with loops, and conditions, and all that. But there’s also now going
to be the aesthetics, the stuff that the user actually sees
and the way you present your data. So we’re going to just introduce
two letters of an acronym today moving forward. Anytime you start writing
logical code in Python, it’s going to be called your controller. It’s sort of like the machine that
operates your entire web application, but anytime you do
something more aesthetic, we’re going to call that your view
code, so C and V being the letters here. And this just refers to
a separation of concerns. Your logic goes in this file. And your aesthetics and
formatting go in this other file. Next week, we’ll introduce databases
in SQL, structured query language. And we’ll introduce an M here,
because the acronym that’s in vogue for quite some time
now is MVC, though there are alternatives to this mental model. And that just means there
are three different types of things you should be thinking
about when building an application. But today we’re going to focus on
these two, controller and view. And up until now, we’ve pretty much
been doing entirely controller stuff when writing code. So the motivation, ultimately, will
be to get to the point of building, essentially, this, the freshman
intramural sports website, which I was the first one to
make back in 1996 as a sophomore, maybe 1997 as a junior, after
having only taken CS50 and a follow on class CS51. And even though this is horrifying
to see nowadays, underneath the hood was a whole bunch of controller logic
and a whole bunch of model code, even though I don’t think
MVC existed as an acronym back then till humans
figured out the pattern. But what it did have via the menu up
here was a whole lot of functionality. So no longer did you have to walk
across campus filling out a form and drop off a sheet of paper in
an RA or a proctor’s dorm room. You can instead just go on the web,
which all of us surely take for granted today, fill out a web-based form. And then at the time, the
proctor in charge of froshims would get an email confirming who
had actually registered for a sport. And eventually we added CSV
files, comma separated values files like spreadsheets, in
which the data was saved. Then we had this really
cool tournament bracket thing, which was all very dynamic
and impressive I’m sure at the time. But we’ll focus on really
just the fundamentals today. So how do we get there? Well, let me go ahead
and open up CS50 IDE and propose that if we want
to make a website, a web application really, and by application
I mean something that does have logic. By website, it’s generally
something that’s static. So that’s what a web app is. It’s something that changes based on
who’s using it, when you’re using it, and what you’re doing with it. Let me go ahead, and in the most
annoying tedious way possible, implement a web
application using Python. So I’m going to go ahead and open a file
called serve.pie to serve up a website. And I’m going to go ahead
and import some library code. So from HTTP server, import something
called base HTTP request handler and also something called HTTP server. And then I’m going to go ahead
and use this keyword class, which is HTTP server, server request, request
handler, base HTTP request handler– handler. Then I’m going to go ahead
and in here implement a function called def
do_GET, which implies– this is a function in Python that
should be called anytime a user visits my web server via
GET, the type of verb that we’ve talked about
in the context of the web. So suppose that my only
goal here is to make a web-based application that for
the moment just says hello world. We know how to do this with HTML,
but with HTML that’s just a file. Let’s write a program that
not just prints hello world, but generates a web page
containing hello world. So the way I might do this is this. I can literally say self, because self
refers to the web server in this case. And I can go ahead and send
a response code of 200, just kind of presumptuously
assuming everything’s going to be OK in a moment. Then I’m going to go ahead
and send a header, which recall, we’ve discussed
briefly in the past, whereby this header is going to
be content type and its value is going to be text HTML,
which is a clue to browser that it’s receiving not a GIF,
or JPEG, or something else, but an actual HTML page. And then I’m going to go ahead
and say that’s it for the headers. I’m going to call function
called end headers. And then lastly, I’m
going to use Python code to dynamically generate my website. Now, what does the website have to have? Just hello world at the end of the
day, but there’s a whole bunch of html we need to do in order
to get to that point. So I’m going to use a function
called wfile write in order to write out the following. Let me go ahead and write out doctype– whoops, exclamation point doctype HTML. Then let me go ahead and do the same
thing, wfile.write, let me go ahead and copy paste this moving forward,
which, of course, is always a bad instinct. Then let me go ahead and output
HTML lang equals en for English. And notice I’m using single
quotes inside my double quotes so that Python doesn’t get confused. Then let me go ahead and
output head tag here. Then what comes after
the head tag typically? Yeah, so usually title lately. So title will be like hello
title, close the title tag. And now we’re going to go ahead
and what comes after title? Close head, I think, if
I’m doing this right. And then after this we
probably have a body. And then oh my God, this is the
worst way to build a website. But let’s go ahead and now say
something simple like hello world. And now let’s go in here
and say something like body. And then finally, let’s go ahead
and end the page and say slash HTML. Done. OK, and now I actually need to configure
the server to listen on a port. So let me go ahead and define a TCP
port of 8080, which we’ve been using. Let me go ahead and define
the server’s address as being, oh, which is what the IDE uses
by default, like a lot of servers. And then let me create the
web server, HTTP server. server_address HTTP server request
handler, httpd.serve_forever. OK, that is how you make a
web-based application that dynamically generates HTML. In retrospect, I really regret typing
all of that out because the whole point now is to throw this code away. Like, this is why frameworks exist. If my goal quite simply at hand is
to write Python code that dynamically generates HTML, then calling
lots of functions like write, which is essentially the equivalent of
print in this context, is just tedious. I got bored writing it, I
accomplished so terribly little at the end of the day. And so frameworks exist
to make our lives easier. But what’s going on underneath
the hood is exactly this. When we start using this
framework called Flask, it’s just going to make all
of this automated for us. It’s going to make it easier to
specify the IP address that you want to run your web
server on, it’s going to make it easier to specify the
port number that you want your server to be listening on, and
it’s going to make it easier to respond to get requests
because Flask will take care of the implementation of some
function like this one called do get. So all of this is there
underneath the hood, but the flask framework gives
us essentially an abstraction on top of that. So what do I actually mean by that? If we want to distill this now
into a simpler web application, first let’s go ahead and do this. Let me go ahead and open up a terminal
window and let me go into my code here and go ahead and
run Python of serve.py. And you’ll see nothing seems
to be happening just yet. So I’m going to go to
cs50 IDE web server to open up a tab containing the
web server that I’m now running. And you’ll see that’s it. That’s the only thing I
accomplished right now. It’s not dynamic to be sure, but
there is code and Python code there with which I could actually
do something dynamically. But let’s instead do this now
with Flask, this framework that seems to make our lives easier. I’m going to go ahead and make a
program called application.py, which is just a convention. I could call it anything I want. And I’m going to go ahead
and say the following. Let’s first go ahead and import
this framework called Flask. And specifically import
capitalized flask, which is the name of the framework itself. And then let me preemptively
import a couple of functions called render template and then this
special global variable called request. You would only know to do this
from reading the documentation. But now let me go ahead
and say, hey Flask, could you please give
me a web application? And this is copy paste. You need this at the top
of any Flask application, and it just means turn this file– application.py– into a web application. Then lastly, I’m going
to go ahead and do this. I’m going to tell Flask I
want to build an app that has a route that’s listening for
slash inside of that virtual envelope. And whenever, Flask, you see a
request for slash from some user, go ahead and call this function
which I arbitrarily called index, but I could call it foo,
or bar, or whatever. And what I want you to do
is this, return hello world. And that’s it. So all of those other lines I
tediously wrote out a moment ago are now distilled into just
7 lines, which none of which are completely obvious
how they work just yet. But if you assume that this means
give me access to the library, turned my file into a web
application, and listen now for get requests on slash,
it kind of makes sense and fits into the mental model that
we introduced a couple of weeks ago with HTML and CSS itself. And to run this server, what I’m going
to do now in my hello directory, which is where a online copy of this
is on the course’s website, I’m going to go ahead and
quite simply say Flask run. So Flask is not only a framework
or code that you have access to as free and open
source code, but it’s also a program you can run at the
command line that also figures out how to turn the server on
and just run it forever. You’re going to see some diagnostic
output at first glance, most of which you don’t have to care about. But there will be a few
URL that’s spit out which is based on your own user name. Mine today is jharvard 3 just
for demonstration purposes. And if you click on that
URL and then click Open, you’ll see now this version of the
same application doing just this. Now, I’m kind of cheating, though. Because if I open up Chrome and view
my page source, notice that of course I’m cheating because all I’ve
done is print out hello world. And if I view the
source of this page, I’m literally only going to say
hello world and no actual HTML because I literally didn’t
type out any HTML in my file. But it turns out that Flask
makes this easy, as well. So let me go ahead and stop the
server here for just a moment. Control-c is your
friend, it gets you out of whatever program is actually running. And let me go ahead and do this. Let me go ahead and not just
return quote-unquote hello world as a hardcoded value. Let me go ahead and return the rendering
of a template called index.html. And so this is a feature of Flask, too. If you want to separate your logic– your controller– from your HTML– your view– you literally put
the former in application.pi, and you put the latter in
an HTML file like this. And you tell the controller code– application.py– show
the user, to render for the user the index.html file. So where do I put this? The convention would be to make
a directory called templates. And then in that directory, go ahead
and put a file called index.html. And if I go ahead and open
up the file that I already created in advance of class here, let
me just show you what this looks like. And then we’ll take a
look at it in a browser. Here is HTML now with
a pretty fancy feature that we’re about to
reveal the utility of. What jumps out at you
is new in this file? AUDIENCE: Two curly braces on the name. DAVID MALAN: Yeah, two curly braces. And somehow or other that
used to be saying world, but in my final version
of this I actually am hinting at some more
powerful capabilities. This kind of looks like a
placeholder, if you will. Maybe someone’s actual name. And here’s where the power of
something like Flask comes into play. It makes it really easy
to do something like this. So let me go ahead
and actually turn this into something a little
more interesting. Let me go into application.py. And before I actually render this HTML
file, let me go ahead and do this. Let me go ahead and declare
a variable called name. And let me go ahead and
check the incoming request, the inside of that virtual envelope. Let me check its arguments or any of
the HTTP parameters that were passed in and try to get something
called name like this. Let me go ahead and save
that and then notice this. Render template takes
multiple arguments. The first one should be the name
of the template you want to render. But if you want to pass
in data dynamically, you can use named parameters in Python. Recall from last week that you can
do things like this, x equals y, and z equals w. You can pass in the names of
the things you want to pass in and their corresponding values. So if I want to pass in a name variable
and set it equal to the name variable that I just defined,
watch what we can do here. Let me go back to my console. Let me go ahead and rerun in my hello
directory, which is available online, Flask run. And now let me go over to this where
it previously said hello world. Let me now, just like with our
Google example a couple of weeks ago, type in not q equals cats,
which is what we did last time, but maybe name equals David
to simulate a get request. And if I did everything right, when I
hit Enter I now see this dynamically. And if I change this now from
David to, say, Veronica, of course, this is going to
dynamically output this. Can someone now try to break my code? Propose an input that I should try
to see if I messed up somewhere. AUDIENCE: No entry. DAVID MALAN: OK, no input. Or what’s that? say again? Name equals name, I like that one, too. So let’s try that, name equals name. So OK, I mean it’s kind of maybe like
a grammatical bug or semantic bug, but not really a code bug per se. That’s just user error. But what if I just get rid of it? OK, that just looks a little stupid. So an aesthetic bug, but we should
probably start handling this. What if I get rid of name altogether? Interesting. It seems that my final version actually
has some built in functionality. So where is that coming from? Well, what if I did this? It turns out that I could
say something like this. If not name, then go ahead
and set name equal to world would be one way of doing it. Or I could actually
use the function here. Turns out that this get function
can take a default value. And so if you read the
documentation, you’ll see that the second
value you provide will be used if the user hasn’t provided one. And so indeed, if I reload now
and see nothing, I get world. And if I instead do name equals, say,
Brian, I get that dynamic output. And so when I say web application,
this is just a hint of what I mean. This is dynamically generated content. It’s not hardcoded, because it’s
actually coming from the user. So when Google implements its
slash search web application, this is the kind of thing they’re doing. It’s way more involved, of course. They’re searching a database, looking
for cats, and dogs, or whatever it is you’re searching for
and then generating HTML. But notice with just
this simple approach can we ourselves generate
any HTML we want dynamically. Because all we have
to do in that template called index.html is exactly this. Hello, comma, and then
a placeholder where name is the variable you’re passing in. And so to be clear, it doesn’t
have to be called name. I could do something like foo,
which would be a little nonsensical. But if I do that, the variable I
plug in needs to be called foo here. And so there’s a one
to one correspondence between the things before the
equal signs and where they get plugged in down here. Any questions, then, on this
simple example, but building block? Yeah? AUDIENCE: [INAUDIBLE]
dynamically generated. DAVID MALAN: Sure. By dynamically generated, I mean I have
not written in advance by typing it out manually a web page that says
hello David, or hello Brian, or hello Veronica. Those pages are all generated
dynamically based on user input. I wrote most of those pages. I wrote everything up into and
after the comma, but then– sorry, up to the comma, but then the
names are dynamically added. Good question. Other questions? All right. So why don’t we rewind
to 1997 or so and see if we can’t build a more dynamic
web application that actually allow students to register
for something that’s a little more compelling than
just providing their name? So let me go ahead and open up froshim0,
which is the first larger scale application we have here today. And notice that I have a few files. So already things are going to
escalate quickly whereby we’re going to suddenly introduce
multiple templates, but we’ll do this in
order to solve a problem. But first let me go ahead
and open up application,py, which just like your main function
in C is kind of the entry point now to a web-based application. So notice that– let’s start like this. Let me go ahead and delete that
and start from the beginning here. Let’s go ahead and do this. In froshim0, we have this
default route of slash, and notice that it’s going
to render index.html. So when you start to
read someone else’s code, you kind of follow these breadcrumbs. So let me go ahead and open
the same folder, froshim0. Let me go into– let me go ahead, rather, let’s
do this from scratch, actually. Let’s do this. Index.html, and let me do
the familiar doctype HTML. Then let me go ahead and do an HTML tag
here, it finishes my thought for me. The head tag here, the title tag
here, froshim0 will be the title here. Let me go ahead and create a body. And now for this web
page, I want to go ahead and have a few things via which the
user can actually register for froshim. So let me go ahead and
have just some title text here, like register for froshims
like I did back in the day. Then let me go ahead
and start a form tag. And then in here, what
information might a student want to provide when registering
for something like a sports team? AUDIENCE: Name. DAVID MALAN: OK, the student’s name. So input type equals text. The name of this input probably
shouldn’t be something generic like q, it should probably be more
descriptive like name. So its a little weird
looking, but name equals name. And we’ll go ahead and do this. And if we really want
to be fancy, we can do like a placeholder text of name just
to in light gray text show the user what we want. And then back in the day,
minimally the students registering for sports
had to provide their dorm, so the building in which they lived. So in HTML, we’ve got
a bunch of input types. We’ve got text boxes, turns out there’s
text areas which are even bigger, check boxes, radio buttons, what’s most
apt perhaps for choosing your dorm? AUDIENCE: Dropdown list. DAVID MALAN: Like a dropdown
list, otherwise called a menu. But which tag? AUDIENCE: Container? DAVID MALAN: Not a container, a
little more precise than that. With what tag can you
generate a dropdown list if you’ve done this before in HTML? AUDIENCE: Select. DAVID MALAN: Select. So it’s not perfectly clearly named,
but it’s, indeed, a select menu by name. And so I can actually do this. Select, and the name of this
field will be dorm, for instance. And then inside of this I’m going
to go ahead and have a few options. So one option might be
let’s say Apley Court, which is one of the buildings
in which freshmen live. There might be another
one called Canoday, and then there’s going
to be bunches of others. And then notice, too, if you’ve never
used a select menu, which you wouldn’t have really had occasion to
yet unless you’ve done this before, these options also have to
have values, which for my purposes are going to be exactly the same. But whereas what’s between the
tags is what the human sees, it’s what’s between these
quotes as the value of value that the computer actually sees. So these are the words that
populate the dropdown menu, these are the values that actually
gets stuffed into the virtual envelope that the student him or
herself actually see. So let me go ahead and save this. Let me go ahead and
now open up my console. And I’m going to borrow
a little code just so that we can do this from scratch here. So let me go ahead and grab from
froshim0 my application.py file and go into my workspace. So let me go ahead now
and run Flask run wherein I have this application.py file. I’m going to see the URL
at which my code now lives. And if I open this up, I’m going
to see an internal server error. So intended at some point because
internal server error, recall, was one of the more arcane status
codes, 500, that you probably have not seen unless you’re visiting
a website where something has actually going wrong. So how do I begin to figure
out what has gone wrong now that I’m actually writing code and
not just writing hard coded HTML? Well, all of the clues
are going to be found here inside of the console window. So when you’re running Flask you are
running a full fledged web server. You are listening via TCP/IP for
incoming requests from users. And so what you’ll see in the console
is not just the diagnostic output when you first start Flask, but you’re
going to see all of the error messages that might actually happen thereafter. And this is a little cryptic looking. Frankly, it’s overwhelming to
see this text at first glance. But whereas in Clang
and in C it generally helps to look at the very top,
sometimes the error messages here in this context of Flask
are kind of toward the bottom. And here we have template not found. Template because it
can’t find index.html. And that’s just because I screwed up. So let me actually exit
Flask by typing control-c. And if I type ls in my directory,
notice that I haven’t quite practiced what I’ve preached. It’s perhaps a little subtle,
but I haven’t organized myself in the right way. What have I done wrong? Yeah? AUDIENCE: Didn’t make
a templates directory. DAVID MALAN: Yeah, it’s
kind of a silly mistake. I didn’t make a templates directory. So you can do this in
a few different ways. By the folder icon up here,
you can create a new folder by right clicking or control clicking. Or in Linux you can do make
dir for make directory. And so I can do make
dir templates enter. And then I can move my
index.html file, which I wrote a moment ago, into my templates
directory by just using mv for move. And now I can go ahead and run
Flask run, cross my fingers, go back to the browser
tab and click reload, and voila, now I actually see the form. So have these kinds of instincts. I didn’t actually intend that,
but I forgot to create the folder. I got this server error. You don’t have to just
stare at the browser, which is not going to have much information. But if you look at the
console, the terminal window that you have control over will you
see those clang-like error messages. So here we have a basic HTML form. It’s not complete because
I didn’t bother typing out all of the dorm names, but I
do have an input of type text as well as the Select menu. And I’m also kind of missing
a key detail, it seems. What should I probably add to this form? AUDIENCE: Where you’re selecting. DAVID MALAN: Well, I’m selecting– I could be selecting dorm, so I
could clean this up in a couple ways. I also am missing a Submit button. Now, it turns out you
could probably hit Enter and it would actually be submitted
by default. But we can fix this. So let me go into index.html. Let me shrink this tab just a little
bit and let me fix both of these things. So let me go ahead and open
up this latest version, which is now in that templates directory. Let me go ahead and at the bottom
here do an input type equals submit, the value of which is going
to be a register, just to make clear to the human what’s going on. Let me go ahead and go back
to my form, click reload. And nothing’s happened just yet. And that’s because by default when you
make changes to some of your files, Flask is not going to notice. And we’ll fix this actually
in the coming problem set by adding more
code and a little more complexity to automate this for you. But when in doubt, just
control-c to quit flask. Then go ahead and rerun
flask, that will reload all of your HTML, all of your Python code. And if I now go back here and click
reload, we’ll see the Register button. So there should never be any surprises. And if there are, just try to
get to the diagnosis thereof. This is also a little unclear,
too, to what this menu is. So it turns out that if you actually
create a bogus option like this that has no value and say something
like dorm, you can save this. Let’s go ahead and restart
Flask and reload the page here. You’ll see that now you see dorm. Unfortunately, this is kind
of stupid because now dorm is where you can
literally live apparently, which doesn’t quite feel right. So there’s HTML fixes for this too. I can actually go in here
and technically say disabled, so you can’t actually select that. Now if I rerun Flask and reload,
now it still shows it there. But because I already selected
Apley Court, you can see it in gray. And we can actually be
a little more specific, if you want to not only disable it, but
select it by default and then go ahead and reload the page here,
now you’ll see hopefully what’s familiar on most websites. It says dorm, but it’s disabled. And even though the
silly checkmark is there, you’re forced to choose an actual dorm. So these are minor aesthetics,
but the kind of things you now have control over. So what’s going to happen? I haven’t actually specified
where this form should go when I actually register because I’m
missing some key details on the form tag. And we haven’t done this
in a couple of weeks. But when we were playfully
reimplementing Google, what else did we add to the form tag? What attributes? Anyone remember? Yeah. Oh, say again? Action, which means what? AUDIENCE: What to do. DAVID MALAN: What to do. All right, so the
action we want to take, even though this is not
necessarily perfectly well named, is where do you want
to submit the form to? And I could actually submit
this in a few different ways. I’m going to go ahead
and say, you know what, submit it to a reasonably
named route slash register. Google, for instance, might
have code that instead says to submit their form to slash search. But we’re not searching for
something, we’re registering. So the name of the route
is entirely up to us. And via what method should
we submit this information? What are our two options? AUDIENCE: Get or post. DAVID MALAN: Get or post. Get tends to be the default
if you don’t mention in links. Why might you want to use post instead? AUDIENCE: You want it to go to a
database in order to do something. DAVID MALAN: Yeah, you want it to
go to a database and do something. And the right verb to use
there is to post information as opposed to getting information. And even more specifically when it
comes to privacy, when you use post, the information doesn’t
end up in the user’s URL, and therefore doesn’t end
up in his or her history, and therefore doesn’t end up in a place
that siblings, or roommates, or anyone else can actually see
just by poking around. So we’ll, indeed, go ahead, because
this is my name, and dorm, and maybe my phone number, and email,
and credit card number, and the like on some other website. I’m going to use post for that instead. So the catch here is
that this week, we now have the ability to
implement slash register. Two weeks ago we could just send
people to Google’s slash search, but now we have the ability
to make our own routes. But how many routes have we
defined thus far in application.py? Just the one. And again, some new syntax with
the funky at sign and the route keyword here. But let me actually
just intuitively guess that if I go ahead and say
app.route slash register, I bet I could implement a second
route and tell the web server to listen in two places, on slash
as well as on slash register. But if I wanted to listen specifically
on post, it actually has to be this. Methods equals quote-unquote post. Because by default just for
convenience, it assumes only gets. So I need to add that. You’d only know that
from the documentation. Now I’m going to go ahead
and define a function. This is slightly an
annoying feature of Flask. You have to give the function a
name even though you’ll probably never reference it anywhere. So I’m going to go ahead and just
reasonably call this register. And now I have a to do. What am I going to want to do when
the user clicks that Submit button? AUDIENCE: Store it. DAVID MALAN: I want
to store it somewhere. So I probably want to store it. But what might I want to do first
before he or she is actually allowed to register, even though
they’ve clicked that submit form? Maybe confirm their information, right? Because if a lazy user comes in, or if
a user accidentally clicks the button or hits enter, they might actually
submit nothing– no name, no dorm. That’s not useful information. It could just be perceived as spam. So we probably want to ask
some kind of logical question like if they didn’t give us a name
or they didn’t give us a dorm, yell at the user in some way. So I’m going to go ahead and do that. So let me actually
express this in Python, kind of like we did last week
with some error checking. So recall that the user’s name can
be gotten from an HTTP parameter. From the request.args get, and
then ask for the name parameter. Their dorm, meanwhile, can come
from request.args get dorm. And again, this
request.args is something we gave ourselves access to up
here when we said, hey Flask, please give me access to the user’s
request, that virtual envelope. Request.arg refers to all of
the HTTP parameters in the URL. And get is just a function or a
method built into that special Flask feature that allows us to get
a specific parameter like name, or dorm, or anything else. And recall that in Python, what’s kind
of nice is that you can say pretty English-like, if not name or not– not dorm, let’s go ahead
and reprimand the user. For instance, we could say failure
because he or she did not actually cooperate as we intended. Otherwise if they did
cooperate, I’m going to go ahead and render template success. And we’ll flesh this
out in just a moment. So I’ve got two scenarios handled. If they didn’t cooperate or if they
did, render quote-unquote failure or a full fledged HTML template. So now that I’ve implemented slash
register and I’m listening for a route by a post, let’s go ahead and
reload the page for good measure. Type in my name. Not going to tell you my dorm, but
you’re going to notice as much. OK, so now the server has noticed that
I didn’t actually cooperate and provide both a name and a dorm. And so it’s returning to me
just quote-unquote failure. So that’s good because now I know, all
right, I did something wrong clearly. Let me go back let me go ahead
and maybe do Canaday here. And now let me go ahead and register. But, but, but, but, this I know
in advance is going to err. Why? I don’t have a success.html. OK, so let’s preemptively address this. Let me actually go in here
to my templates directory. I’m going to go ahead and create
a new file called success.html. Let me go into the
templates to save it there. And you know what, success
can be a pretty simple page. And let me open my index page, let
me copy that, let me go into success, let me paste this. Let me get rid of all of that body
and just say success, for instance. So let me now go ahead
and go restart Flask, because I’ve made a new template. Let me go ahead and reload the
form just for good measure. Let me go ahead and give you my name
this time, and OK, I live in Canaday, and register. And what did I do wrong this time? So it turns out you can’t
do it this way, obviously. [LAUGHTER] So when you’re
actually submitting information to via a form via get,
Flask very cleverly puts that information in a different place. Because by definition, as I
claim very correctly earlier, in request.args are all
of the key value pairs that are in the URL that are
coming in from the user’s request. But when you submit via post, for
reasons I wish were now otherwise you actually have to access those
values via a different variable. So instead of using request.args, you
have to use request.form both here and here to make clear– and this is
horribly named for exactly the reasons that I think I’m tripping over here– because they actually are both coming
in via a form, via a get or post. But Flask puts get arguments in args
and puts post arguments in form. Thereby leading clearly
to potential confusion. But if I go ahead now and load this
version of the site and I keep hoping, I’m going to go ahead now run Flask,
restart this page, type in David, give you my dorm, and register and
successfully register for froshim. See how easy web programming is? So hopefully now we can at least focus
on the structure of what I’ve done and now begin to improve it. Because notice that I kind of did not
practice what I preached a moment ago. What habit did I violate when
I whipped up success.html? How did I get to that point? Yeah? AUDIENCE: Copy and paste. DAVID MALAN: Yeah, I copied and paste. Which again, usually in programming
not the right thing to do. Might get the job done super fast,
but it’s probably the wrong instinct, because it’s going to get
harder and harder to maintain. Now, why is that? You’ve played HTML a
couple of weeks ago. And recall from that problem set
when you had to make a home page, you probably found yourselves
copying and pasting across your two, or three, or four pages because you
wanted them to kind of look the same. And therefore it made sense for
them to have some commonalities. But in HTML alone, there was no way to
say use the same layout for my whole site– use the same color scheme,
the same fonts, the same CSS– but just change the body of the
page for each individual page. And so some of you very rightly
on discourse and beyond like posted questions asking,
could you do this? And you can’t really in HTML alone. But now that we have access to
Python, an actual programming language they can do things logically,
now you can actually start to factor those things out, too. And notice in this file, success.html,
as well as in index.html, what are some of the
commonalities, suffice it to say? The form is only in
one of them, but what else is obviously redundant everywhere? The title, the head of the page more
generally, the doctype at the very top, the body tag alone. And you could imagine there’d be even
more details like from your own home pages that you wanted to be
the same across multiple pages. So let’s actually take a look at
a refactorization of this code, the one I did write in
advance in froshim0, and you’ll see why it
actually makes sense to have not only multiple
files, each of which represents one of your
routes or your views, but also to have this
file called layout.html. In Flask, when building a
web application that you know is going to follow a certain
structural pattern, commonalities across all of your pages,
you can actually do this. So in this file here, layout.html
is a whole bunch of hardcoded HTML. And it’s pretty simple. It’s got my HTML tag, head tag, title
tag, body tag, and a few other things, but that’s the general
structure of the page. And notice it has this
funky syntax in the middle. In white here is what’s
called the block. This is now Flask specific. Just like Flask supports those
two curly braces on the left and the right that says
put a value here, flask also supports this other notation,
curly brace percent and percent curly brace that actually allows you to put
placeholders for actual chunks of HTML. Not just variables, but
actual chunks of HTML. And so this layout you can
think of as a mold or a template literally that all of your other pages
are going to be structured based on, but they are going to vary in
this line and only this line. And we’re going to put as much HTML
between the body tags as we want, the open and the close tag. This just indicates to Flask this is
the stuff that should be changing. So if I now look at my
index.html, which recall earlier contained my form index.html. Notice that here’s the form,
and I finished it earlier. I went ahead and typed out
all of the freshman dorms, not just the two of them. And you’ll see that the
file starts almost the same and then continues with more stuff. But notice what’s missing
from index.html this time. No doctype. No HTML tag. No head tag, no title tag, no body tag. All of the common stuff
has been factored out. But there’s some funky new syntax
that, again, is Flask specific. This first line is the link
between this file and the layout. That first line says, hey
Flask, this index.html file extends the definition of layout.html. So it says grab that template
and plug myself in there. What do you want to plug in? The same syntax here. When you actually put
stuff between the block tag and the end block tag, which is down
below, that’s when you say to Flask, go ahead and take this stuff and plug
it into the placeholder in the layout. So meanwhile, the success page also
now can be a little more sophisticated. If I go into success,
it’s not very complicated. And honestly, it doesn’t
even look like HTML anymore because we’re using
these more dynamic features. But this just says, hey
Flask, use the same layout so the page is structured
exactly the same. But for the body, go ahead and
plug in this value instead. So indeed, when you go ahead
and load this success message, you see this message
here– not just success, I expounded here and
said you are registered. Well, not really, that’s
because there’s no database yet. But that’s going to generate
a full fledged HTML page. And what about failure? Before I was just cheating and
just saying return failure, quote-unquote, no HTML at all. The failure page is going
to be almost the same, but now I can actually
provide some descriptive text. This body just says you must
provide your name and dorm, thereby admonishing the user for
not having cooperated properly. So now your home pages, if you
kind of extrapolate from this, could have the exact same
layout, aesthetics and menu bars, and all of that fanciness, but only
the content would have to change. And you can get out of the business
of just copying and pasting. So there, too, to your question
earlier about dynamism, the dynamism doesn’t have
to just come from the user. It can also come from the
construction dynamically of a website based on multiple pages. So at the end of the
day, the browser has no idea that Python exists,
has no familiarity with Flask. All the browser still
sees is an HTML page. But what Flask and in
turn Python are doing for us is constructing that
page dynamically, following the rules from two weeks
to go in HTML and CSS, and following last week’s
rules on how Python works. Questions? AUDIENCE: So even though [INAUDIBLE] DAVID MALAN: It’s not. Good question. This new syntax, the double curly
braces that we saw earlier and now the curly brace percent signs, this
is actually yet another language called jinja– J-I-N-J-A– which is
a templating language. And there’s dozens of these things
in the world, people just come up with their own syntax. And the reason for the funky
syntax is that the author of jinja presumably could think
of no other language that uses like a curly brace
and a percent sign and a percent sign and a curly brace. And so therefore they
decided, you know what, I’m going to use this syntax because
it will look distinct from HTML, and CSS, and Python So
that frameworks like Flask don’t confuse it with something else. AUDIENCE: So do you have to upload that
into [INAUDIBLE],, or is it automatic? DAVID MALAN: It’s
automatically supported. So Flask by default supports jinja. It could have come up with
its own templating syntax. But whoever invented Flask decided
I don’t need to reinvent this wheel, someone else already made
a templating language that gives me this functionality. So I’m going to combine
our works into one. And I didn’t call it a
language a moment ago, because frankly, HTML,
CSS, Python, JavaScript– I mean, we’re already
running out of fingers here. But jinja is, indeed,
yet another language. It’s just not a programming
language per se, though it will have some
control flow features that we’ll see in a little bit. It’s just much more limited than Python. Other questions? AUDIENCE: Is it possible to combine
the success and failure HTML files into one just for better design? DAVID MALAN: Good question. Could you combine the success
and the failure pages into one? Short answer, yes. And let me not show it yet, because
it’ll get a little more complicated. But yes, I could imagine passing
a variable in that’s a Boolean– true or false– into just one of these templates. And maybe I call the new
template result.html. I can actually then have an if
condition in my template that says if the result is true, say this. Else if the result is
false, say this other thing. So you could do that, yes. Generally, though, it’s probably
cleaner to keep messaging separate if they functionally do something else. After all, these files
are pretty small anyway. Yeah? AUDIENCE: Just for
question, what does the user see if they were to open up
the debugging console on Chrome and look at at it, what do they
see as the HTML that shows up? DAVID MALAN: Really good question. What does the user see? We can answer this by just
literally opening Chrome and opening View Page Source or the Inspector. This is what the browser sees. So when I claimed earlier that the
browser has no idea of Python or Flask are even involved,
that is, indeed, true. Because what browser’s
receiving at the end of the day is just this, the
dynamically constructed HTML. Good question. Yeah? AUDIENCE: [INAUDIBLE] can you
also put Python code in there, or is it just HTML? Good question. We’ll see more of this
in just a little bit. The question is, can you between
the curly brace and percent signs put actual Python code? You can put stuff that looks like
Python code, but not all of Python. And so more of that in a bit. AUDIENCE: Is there a function
call or something like that? DAVID MALAN: Only certain functions. Templating languages, long
story short, are sandbox so that they are not as expressive
as a real programming language. Otherwise you are vulnerable
to potential hacks. You want their functionality to
be very limited because they’re only about displaying data, not about
thinking or doing logic generally. More on that in a bit. All right, that was a lot all at once. Let’s take a five minute break
here, turn on some music, come back, and we’ll make this act better. All right, we are back. So to recap where
froshim0 left off, we now have this structure which
is pretty much conventional. Any web application
we make here on out is going to follow this pattern of having
an application.py entry point where all the interesting stuff
starts, a layout.html file in your templates
directory that lays out the whole site and any commonalities,
and then one or more other pages that actually represent your individual
views that correspond to one or more of your actual routes. So now we’re at the point
of a stable baseline, but had we dived in right to this, it
would perhaps not make as much sense as to why we did this
various factorization. So let’s now improve this. Because of course, if
you look at success.html, it just claims you are registered. Well, not really. Because in application.py, did we do
anything with the user’s information? No. We just checked, did
they give us information? And if so, we claim success. Else if they missed their name and/or
their dorm, we just claimed failure. So what might a data
structure be in Python where we could store registrants? We don’t have databases
yet, we don’t have SQL yet. That’s a week ahead. AUDIENCE: Array. DAVID MALAN: Yeah, we could
use an array, otherwise known as a list in Python. So let me propose how we might do this. Let me actually open up froshims1 for
our second iteration of this program. And in application.py, notice this. At the very top of the
file, not only am I creating my application using
the same line as before, and I’ve commented things this time
in advance using the hash symbol, notice that I claim that
on line 6 and 7 here, here is an empty list for all of the
students who have registered. This way we can keep
the information around. And we only did this briefly
last time, but does anyone remember how you add
something to a list in Python? By what function? Append. So if you have .append at
the end of a list’s name, you can add something to it. So where is that going to go? Well, here is my route for slash,
implies, again, get by default. That’s the default route
that a human might get, and they are going to see
index.html, which contains that form. If I scroll down now, you’ll see that I
have a register route just like before. But I’m doing one additional step. Which is the new line here, to be clear? Yeah, 26. So I could implement this
in any number of ways. But the key detail is that I
reference the list name– students, but I could have called it anything. .append, as someone proposed, is how you
add something to the end of the list. And then I can add anything I want. To keep it simple, I’m
just going to add a string. And I’m going to keep
it super simple and just say the string is so-and-so
from such and such a dorm. So David from Matthews Hall,
or Brian from wherever. And so here we have placeholders
using f strings in Python. So this has nothing
to do with Flask, this has nothing to do with jinja or
anything we just talked about. This has to do everything with
last week’s syntax in Python alone. So this appends to that list
this name from this dorm. So let’s go ahead now
and try this version out. If I go into my source sub and directory
for today’s code into froshims1 and run flask run, we’ll see
a URL that I can now visit. Let me go ahead and
open that for froshims1. Notice that I have that
complete dropdown now. Let me go ahead and say David, but
I’m not going to tell you my dorm yet and try to register. Now I see a more friendly
message, not just failure. And that’s because of my
new and improved template. OK, I’ll go ahead and be David,
and I’ll be from Matthews here. Let me go ahead and register and voila. Now we see David from
Matthews has registered. And it seems to be all of a sudden
in the form of a new bulleted list. But where did that actually come from? Well, I don’t know. Let me try this again. Let me go back to slash, which is
the route that gives me the form. Let me go ahead and type in not
David this time, but say, Brian. And Brian, which dorm are you in? AUDIENCE: Pennypacker. DAVID MALAN: Pennypacker. So let me choose this from the
menu instead and click Register. And now we see Brian from Pennypacker. So somehow the application
is changing state, and notice the URL that we’re
at is called slash registrants. So that seems to be a third route this
time that apparently is not interactive per se, it just spits out the
list of registered students. So let’s just put the
proverbial engineering hat on. If we go about implementing
this slash registrants route, logically what must that code be doing
in verbal pseudocode, if you will? AUDIENCE: A for loop? DAVID MALAN: Like a for
loop, iterating over what? AUDIENCE: In the list that
saves all the registrants. DAVID MALAN: Yeah. Iterating over the
list of students which contains all of those registrants. And the template, meanwhile, probably
has like an LI tag for list item and a UL tag for unordered list,
which gives me the bulleted list. So let’s take a look at that. So how do we follow these breadcrumbs? Well, if I scroll up in
application.py, we’ll see a route called slash registrants. And you’ll see that
all it does apparently is it returns a template
called registered.html, where registered.html is probably a
template that is generating that list. But there’s something
different this time. I’m passing in an argument. And we saw this earlier. When I wanted to pass in name
equals David or name equals Brian, I just grabbed that from a variable. This time I’m not doing request.args,
I’m not doing request.form. Because what is students? Where did this come from? That’s the list from higher up. Recall that we have this global variable
at the top of the program, students, which is initialized to an empty list. But recall that we keep appending
to it in my register route. So I can go ahead and
say, you know what? Go ahead and pass into register.html
a template– or rather, a list– called students whose
value is exactly that. And again, it’s stupid looking that
you have the same word on the left and the right of the variable name. You could do this differently. Again, you could say foo, you
could say x, or y, or anything. But frankly, it tends to
make most sense, just pass in the same name as the variable that
you care about so that the template can see exactly that. So what’s the next breadcrumb? If I want to understand
exactly what is happening, what file should I open up next perhaps? Probably register.html. So let’s go in there. It’s in my templates
directory by definition, and you’ll see, indeed, a failure
message which allows me to error check. Index, which contains the form; layout,
which contains the overall structure; and finally, registered.html. And now we can answer the question that
you asked earlier about Python code in the template. So this one looks more
advanced than before, but notice it follows a pattern. Register.html extends that same layout. So it borrows from that same
mold, so it looks the same. The body of this page, though,
is just this snippet of HTML. Give me an unordered list,
open and closed, and this is what you can do now with jinja. Again, it’s almost
identical to Python, so you don’t have to worry about thinking
about learning yet another language. It’s just a subset of
Python essentially. So if I want to output in the
list of all of the students, I use my jinja syntax here, my template
syntax with curly brace percent. And I say for student and students. Just like in Python, that induces
an iteration over that list. And then what do I want to output? Well, we can borrow our curly
braces from our name example and just do list item, plug in the
name of the student, close list item. And then endfor. So this is the one stupid
thing with the templates. Whereas in Python proper,
recall that you can just say for student in
students, you have a colon and then indentation handles everything. The problem with that
in the world of HTML is that browsers recall
ignore all whitespace, like whitespace has no special
significance, but in Python it does. So the way people solve
this is you literally, if a little weirdly, say endfor– one word, no space. And that’s it. And indentation helps you
read something like this. So what is the HTML I’m getting back? I can actually look at this. Let me go ahead and view
page source in Chrome, and you’ll see it’s not quite
as pretty as might be ideal because there’s a lot of whitespace
which comes from those templates from my having pretty
printed those, as well. But this is syntactically correct,
and I’m dynamically putting this part inside of this layout. Any questions, then, on this? AUDIENCE: So if we restart
the server, whatever’s stored in the list, that goes away, right? DAVID MALAN: Good question. Let’s kill Flask with control-c. Let’s rerun the server. And let me go back to my
registrants route and reload. And sadly, yes, this is not the best
way to register students for a sport. Because if the server ever goes
offline, loses power, you hit control-c, you obviously, indeed, lose everyone. And notice, too, even
though we’ve generally frowned upon using global variables,
which this students list indeed is, why did I define it
up here in line 7 and not, for instance, in my register route here? Because indeed, I’m
appending to the list here. But I very deliberately did
not declare the list there. Yeah? AUDIENCE: You’re using
it in other parts. DAVID MALAN: I’m using it elsewhere in
my other routes, the registrants route. And also even more to the point,
if I declared a list here, it becomes by definition
a local variable. Which means as soon as
this function exits, now I’ve just thrown away those
students who register immediately not even after a control-c. So this was a better approach
to do it, but it’s not what I did way back in my day. I actually did something
that was a little fancier. So at the time, I didn’t
really know– at least in, what, 1997– anything about databases. I don’t think I even knew about
CSV files just yet, or at least how to create them dynamically. So I instead took this approach. Let me go into froshims2, and it has
noticed the same templates as before. And indeed, I pretty much copied
and pasted for this second example. But in application.py,
notice this fanciness. So here I have almost the same
thing up top in terms of Flask, but I’m also using this OS
library, more on that in a bit. But what about line 2? It’s subtle, but I rattled this acronym
off I think just once weeks ago, SMTP. Does anyone know what that stands for? AUDIENCE: Simple mail transfer protocol? DAVID MALAN: Yeah, simple mail
transfer protocol– email, that is. So Python comes with
built in functionality via which you can send emails,
and this is exactly what I did when I first made this website. Didn’t know anything about
databases, I didn’t know anything about saving things to files
just yet, I was still learning. But I didn’t realize, hm,
I could use programming to send an email to the proctor or
the RA who was overseeing the sports program so that they could
just save it in a folder and know who had registered. It’s not super user
friendly, but it at least got the job done because they were
then able to track everything. So in this program, notice that
I have my route for my form. And I have this register route
but a few new lines of code. And you would only know how to do
this by reading the documentation. But in this case here, notice what
I’m doing in my register route. I’m first getting the user’s name and
their email this time and their dorm. Then I’m error checking. If they didn’t give me a name,
or their email, or the dorm, render failure.html to
apprise them as much. Then go ahead and do
these lines of code. And this is more of a mouthful,
and you would only, again, know this from the documentation. But it turns out if you
read the documentation for this SMTP lib or library, you can
use lines of code like this as follows. You can tell the library what
server to use for sending email. And it turns out if you
read Gmail’s documentation, you can use smtp.gmail.com to
automatically send e-mails not using the web UI, but using code. 587 is the TCP port that they use. So it’s not 80, it’s not
443, it’s 587 by convention. Starttls, if you read the
documentation, says turn on encryption. So the email is encrypted
between you and Gmail. Then go ahead and log in with a
certain username and password. I created an account in advance called
[email protected], and my password is in my IDE’s environment. I stored it elsewhere so that
it’s not visible on screen, otherwise people could
send emails as John. Then I go ahead and call literally
a function called send mail. And if you read the
documentation, this one takes as argument who you
want to send email to, the contents of the email
that you want to send, and the message that you
actually want to send here. Or rather, this is the from
address, the to address, and the actual message
that you want to send. After that, you just go ahead and
render template and assume success. I could add more error
checking, like I should probably check if anything went wrong
here, but I’m keeping it simple. But these new lines that are
highlighted actually send an email. So let’s try this. Let me go into froshims2 and let
me go ahead and do Flask run. Let me go ahead and open
up the page here, slash. And notice I do, indeed, have
a second field for text now. So this will be David,
and this will be– let’s see, how about let’s go
ahead and just register not myself, since it’s not my email account,
but John Harvard who we claim’s email is cs50.net, jharvard thereat. And he lives in say, Weld. Let’s go ahead and click Register. All right, it’s taking a
little longer this time, but it was doing a little
more work sending an email. So now let’s try to go to
gmail.com, open this up. Ooh. In my inbox, you are registered. If I open this up,
notice [email protected] has sent me an email by a BC seed,
at least keep part of the information private. And it just says in the body of
the message if I move the cursor, you are registered. So I did a little more
back in 1997, but I included like the user’s name, and
their email address, and their dorm, and maybe their phone number or whatnot,
and the sports they were interested in. But the idea is exactly that. You can send any information
you want just by now using code. You could not do that with
HTML or with CSS alone. Any questions, then, on this? Yeah? AUDIENCE: Last week when we wrote
code in Python we had to like say if name equals the
function to like execute that. How come in this we’re not doing that? DAVID MALAN: That was all in JavaScript. So that allusion to if name equals,
and then you assign it to a function, I think you’re referring to
our JavaScript examples, no? OK. So we’ll actually come back
to that in a little bit where we reintroduce
a bit of JavaScript, which actually gives us some
more functionality reminiscent of those examples. Other question? AUDIENCE: What email address
did you send that email from, and don’t you need to enter
like a password to make sure that no one just randomly sends email? DAVID MALAN: Yeah, it’s
a really good question. So via what email address did I
send that, and to whom was it sent? So again, this is the from
address, this is the to address, and this is now the message. And just because I only have one email
account open, I had John send himself an email in this case. Theoretically if I were running the
freshmen intramural sports program, I could try to do this and change
this from address to be myself. The catch is that Gmail actually has
protections in place so that if you’ve logged in as jharvard
with his password, then the email, no matter what you
specify as the from address, is actually going to be overridden
to be from John Harvard. However, this does speak to the
potential insecurity of email. If you don’t use Gmail
but you use a third party service that is not so rigorous
with its error checking, it is incredibly easy to fake
emails from one person to another. I mean, look through your
spam folder sometime. Most of those people who send
you those spams don’t exist. Like, the email addresses
and/or the names are fake. And yet they might appear to actually
be from a sibling of yours, a family member, or a friend, even though
those humans did not send e-mails. And that’s because some
spammer has written code like this in Python or
some other language, but has overridden these fields,
but used a server that’s not Gmail that doesn’t enforce these policies. Other questions? Fun fact, also in 1995,
I learned how to send– or how to change the
from address on an email. And turns out at Harvard there’s
this entity called the ad board who doesn’t like doing this. So don’t do that. Sometimes there’s human defenses in
place for this, not just technological. Thankfully, my friend whom I
faked an email from did not– it worked out OK. All right. You have now great power,
don’t use it for evil. All right, so let’s go ahead now and
do another example, but that takes this a further step, adding, finally
some persistence of information. Let’s go ahead into froshims3
now and open up application.py. So recall that we can use CSV
files– comma separated value files– to create the illusion
of like spreadsheets, but now we’re actually going
to create them ourselves. The code for this is a
little more involved, and the only thing I’ve changed
now really is the register method. So in version one of this code, I saved
it in a global list just in memory. That was not good because it
gets thrown away too easily. Version two of this
we just sent an email to the proctor who runs the program. That was a little better, because at
least they can then save the email. Version three, we’re going to use
a very lightweight database called the CSV file that saves it
to my hard drive permanently. So even when the server stops,
the data is still there. So in Python, how does this work? Well, notice that I’ve improved my
register route this time as follows. If the user did not give
me their name or dorm, then I go ahead and render a failure. That’s pretty much the
same logic as before, but I didn’t bother declaring
the variables this time, I just called the functions directly. Here’s a new line of code that might be
reminiscent of some of your past file I/O code. In line 16 here, I’m telling Python
to go ahead and open a file called registered.csv quote-unquote a. So we’ve seen R, we’ve
seen W for read and write. Anyone recall what a is, or no? AUDIENCE: Append. DAVID MALAN: It happens to
mean append, which means just add a row to the file, which is nice. Because if there’s already students
registered, when a new one registers we just want to append to
the bottom of the file. Quote-unquote a is supported by
Python with this open function. That gives me back a reference
or like a pointer to file, even though Python
does not have pointers. Then this is the new feature. And here, too, you’d only know
this from having seen an example or you reading the documentation. You can use the CSV library, which
we’ll see as imported up above. And you can ask for a writer, a piece
of code that writes out– that is, creates– CSV files. And specifically, you want
to write to this file. That library– the CSV
library– comes with a function called write row, which
does what it says. If you pass it in a comma
separated list of fields that you want to represent your
first column, your second column, and your third column, it will
handle the writing for you so you don’t have to get
into the weeds of file I/O like you did several problem sets ago. Notice the subtlety. You do need to have these
second pair of parentheses, because technically what
you’re giving it is a tuple. We talked very briefly
about that last week, which is just like an x comma y
pair, or latitude comma longitude. Same idea here. First column, second column, and
so forth is a so-called tuple. Then I close the file,
then I render the template. So what does this actually do for me? Well, let me go into my
folder froshims3 here. And notice register.csv
at the moment is empty. This is a CSV file. Nothing’s going on inside of that. There’s no one registered yet. But let me go ahead, then, and
go into froshims3, run Flask run. Let me go ahead and load this up. And you’ll see the same kind
of form, but also a new link. Notice that no one’s registered
yet if I click on that link. But if I go into here and register
David from Matthews and click Register, now it claims I am registered really. Let me click this link and notice
that it’s very small on the screen, but slash registered is
where this is going to lead me, which is just where I was before. You see that now David
from Matthews registered. Let me go back to the form itself. Let’s register, say,
Brian from Pennypacker. Click Register. He, too, is apparently registered. Let’s click that link. Brian from Pennypacker. All right, so where is this data going? Let me go back to the IDE,
close my registered CSV file, because it’s probably
changed and open it up. And voila, indeed, it’s
prompting me to reload it. There is the file. And notice David comma Matthews, Brian
comma Pennypacker, all of those rows were written out for me. So now I actually have a database. And even though it’s kind of a simple
database, you know what I can do? Let me go ahead and right click
or control click on it in the IDE, download it into my Downloads folder. And then if I actually
open this thing, if I have Excel installed
or Apple Numbers, which is the first time I’ve ever used it,
let me go ahead and open that file. Opening register.csv. And voila, here now is a file. And Numbers is formatting
it in kind of a funky way, but it is showing rows and columns. Those of you who are more familiar
with Excel we can do that, too. Let me go down here. Let me go into my Downloads folder. Control click or right click here, and
this time open it with Microsoft Excel. And if you’ve seen Excel before,
we’ll probably see a very similar UI. Because anytime Excel or Numbers– OK, first time I’ve used Excel. So that, too, will open up
some rows and columns, as well. So CSV files are just very
lightweight spreadsheets. But what’s cool about them is that
you can create them so easily. You just have to put commas in there. Now, as an aside, can you
contrive a user’s input that could potentially break a CSV file? What could a human type in that could
potentially break your own CSV files? A comma, right? If it’s like David Mayland comma
junior, or something like that. Or anything with weird punctuation. This is why you use libraries. That CSV library in
this code, which we’re importing at the very top of
this version 3 of the code, is actually handling all
of that complexity for us. When the library encounters David
Mayland comma junior if that’s the user’s input, it
will then additionally put quotes around my whole
name, thereby making sure that my comma is inside
quotes and not, therefore, confused with the special comma that
demarcates the start of other columns. So again, that’s why you
don’t reinvent the wheel, because corner cases like that arise. Well, what about slash registered,
which is this list that’s generating an unordered list? Let’s see how that works. If I scroll down to this
code, notice that it’s not just a simple matter of
grabbing a global variable, because there is no
global variable anymore. Now I have to read it
from that CSV file. So here’s three new lines of
code that work as follows. I’m going to go ahead and open this
file, register.csv, in Read mode this time, not append. I’m going to go ahead now and say
hey Python, use the CSV reader– which is the opposite of writer– on that file. And then, hey Python, go ahead and
turn that whole reader into a list. So you’d only know this from
reading the documentation. It turns out this is the recommendation. If you want to take a reader and
just read the whole thing into memory at once and convert it to a Python
list, you literally just pass it to this list function. That gives me a list I’m
going to call students, and then I can do my
same code as before. For good measure, I
should probably do what I did last time, which is file.close
to close the file, as well, just to make sure it’s closed
the next time it’s opened. But I can actually simplify this,
and you’ll see more of these examples online. It’s actually more conventional
in Python not to do this, but instead to change
your code as follows. To not bother closing it explicitly,
to instead use a keyword called with to instead put the variable name back
there and indent everything underneath. Doesn’t matter for our
purposes which one you do. The first one was correct. The second one is correct. This is just more conventional,
if only because it handles the closing of the file for you. So if you see this online, that’s
all that’s happening there. But it’s just like in C doing fopen and
fclose, or in this case open and close like I had a moment ago. Any questions, then? AUDIENCE: How would
you handle duplicates? DAVID MALAN: How would I duplicates? good question. So haven’t handled duplicates here
at all, and David from Matthews could register again and again. But logically what might I do? Well, it probably belongs
here in my register route. I probably want to do more error
checking than just these two lines. Because what I probably want to
do to see if David from Matthews is already registered
is open the CSV file, iterate over its lines looking for
David and for Matthews on the same line and then show a failure to
the user if he or she is trying to register for a second time. I’ve not made it easy with
this code, and frankly that’s going to be so much
easier next week with SQL. SQL, this other language
for databases, will make it easy to search data
that has already been saved. CSV files do not make this easy. It’s doable, but you have
to write more lines of code. So more on that to come. Other questions? All right, so let’s skip
ahead to one final example, froshim6, which we’ll do something
a little more for us here. So if I go ahead into
froshim6, notice that if I do Flask run, and go back to the
website here, and reload the screen, and I go ahead and give you my name, but
no, I’m not going to give you my dorm, we have this feature. It’s ugly again, but where did
we see this kind of functionality when the user does not cooperate? Or how did I implement this, apparently? AUDIENCE: JavaScript? DAVID MALAN: Yeah, JavaScript. So it turns out that with
Python, you can obviously validate the user’s input on the
server by just checking what’s in request.args or request.form and
then yell at the user success or failure accordingly. But you can also use JavaScript– and
honestly, we did this two weeks ago, so we just seem to be solving
the same problems again. So how do you think about this? Should I be checking for the
user’s name and dorm in JavaScript? Should I be checking for the
user’s name and dorm on the server? I mean, mixed messages now. AUDIENCE: Whatever’s fastest. DAVID MALAN: Whatever fastest. That’s a pretty good heuristic
to use, what’s fastest. And we can make it prettier by
using Bootstrap or some library to give you like a colorful box, or
red error text or something like that. So which probably is faster,
Python or JavaScript? AUDIENCE: JavaScript. DAVID MALAN: JavaScript. Why, is JavaScript just a
better, faster language? AUDIENCE: You’re not
creating it [INAUDIBLE] DAVID MALAN: Say again? AUDIENCE: You’re not
creating it on a new server, so it’s all happening on the same– DAVID MALAN: That’s why, yeah. We don’t have to get into the
religious debate of which language is better or faster, but where they’re
running is certainly important. JavaScript is running, recall,
by definition, in the browser. It is sent as JavaScript
code to the browser which then executes it client side. Python by definition today is
doing everything server side. And indeed, the browser
doesn’t even know Python is involved, because all it
gets is the HTML code that results. So OK, that seems to
be an argument for not doing all of the new work we did
today with if not name, if not dorm, and all of that, and
just use JavaScript. But the problem is that if you get
a little sloppy or a little clever and only implement your
error checking client side. Because as you say, it’s faster,
and frankly once I make it prettier, it’s just going to be more
interactive and more seamless. The problem is you can’t
and should not trust users. Suppose that I’m a
malicious user and I just want to inject some bogus data into
your website, or I want to spam you, or subscribe 1,000 freshmen who
don’t actually exist, or just generally create problems for you. Well, you might think,
well, that’s OK, I have some defenses in
place, and JavaScript code, and this adversary is going to get
very quick feedback, very pretty feedback that they’ve not
provided these various fields. But honestly, you can’t trust
anything ever coming from the human. If I open up Chrome’s
developer tools, and I go to this down here to the dot
dot dot menu, and I go to Settings, and I go down here, there. That’s all it takes to
disable all of your hard work. He or she can just
open up their browser– Chrome or something else– turn off JavaScript. So now when I actually
submit this form, there’s going to be no on-submit
checking, no on-click handling. All of that is disabled. So if I go ahead and
click Register, I at least still, in this version of froshims,
have server side checking, as well. So this might be a little frustrating,
but it’s kind of the reality. It is perfectly fine
to use JavaScript code and use client side code to give the
user a better experience– a.k.a. UX, user experience. But you can’t cut
corners and not implement the same kind of logic
server side because you need to defend against this. Otherwise bogus data is going
to end up in your database, things are going to go wrong. Never, ever, ever trust the user. Any questions? AUDIENCE: Can you do the same via CSS? DAVID MALAN: Can you
do the same with CSS? AUDIENCE: Yes, can you [INAUDIBLE]
JavaScript [INAUDIBLE]?? DAVID MALAN: Not with CSS alone. You can use CSS to make the
error messages far prettier, yes, but not logically enough with CSS alone. And in fact, just to give you a sense
now how you can make things prettier since I keep alluding to better design
skills than the ones I’m showing here. If we go to Bootstrap, this very
popular, free, and open source library for CSS, it actually has
some interactive features, as well. And if I go under components in
the documentation and I scroll down and I go to forms,
you’ll see, one, notice that these forms are already
way prettier than the ones I’ve been making, right? It’s like black text, and
gray text, and small text. It just looks nicer and cleaner. But it’s relatively easy to do this. And indeed, for the next
problems that you’ll be welcome to copy and paste some
of this sample code and HTML, use Bootstrap CSS just to
make your forms prettier. But what it can really do if I
go to the sub menu over here, notice that there’s this
validation section in Bootstrap. And other libraries have this, too. And you’ll want to read
the actual documentation. But if I just scroll down for a while,
here’s a sample form in Bootstrap. It already looks a little prettier than
anything I’ve made in just raw HTML. But notice if I don’t cooperate, you
can do really cool validation of forms with green and red text if
the user does or doesn’t cooperate by using a mix of CSS,
but with some JavaScript code. And so what Bootstrap does
for you is it actually automates some of the process
of that JavaScript code that we saw two weeks
ago and just used now. But it doesn’t just pop
up a message for the user, it actually gives them
more immediate feedback. And almost any popular web site
you visit these days gives you this more immediate proximal input. Generally you don’t see some
simple error message popping up, even though that’s easier to do. Any questions? All right, so where did
that logic come from? So let me go into, for
instance, my template file now for the form in froshim6– again, the last of these examples– and you’ll notice that I did this. If I scroll through this
file, you’ll see the same HTML as we’ve been using for some time. But notice at the bottom of the
page that I draw some inspiration from two weeks back when we looked
at HTML, and CSS, and JavaScript. So just as a quick refresher,
notice how this is working. This line of code says
in JavaScript check the document using the query selector,
which lets you select anything in the web page looking for a form. When that form is submitted, call
the following anonymous function. If the document query selector finds
an input that does not have a value– and I say not because of
the exclamation point here– then yell at the user with
this, you must provide your name and return false. Else if the user did not provide a value
for the select menu– a.k.a. the dorm– go ahead and alert them that
they must provide the dorm, otherwise return true. And just to be clear and to
recall from two weeks ago, what am I returning false? What does that effect have logically? Yeah, say again? What was that? I heard a whisper here. No? Yeah, it prevents
submission of the form. The default behavior of a form
is it wants to be submitted. That’s why they exist. But if you return
false in JavaScript, it will short circuit that and
prevent that default behavior, thereby stopping the user from
submitting the form at all. So let’s take one step back now, there’s
now so much going on in this one file alone. In this sixth and final
example, notice that we have application.py, which is the
entry point, the so-called controller of this web application. It has a route which very simply
for slash looks like this. When the user gets slash, this
template is simply returned. What is in that index.html template? Well, it contains a partial HTML file. It contains this HTML. But it does not contain the doctype,
the HTML tag, head tag, the body tag, the title tag, and all of that. It only contains the stuff that
should go inside of the body tag. Because this file is
using a bit of jinja, which is the templating
language that Flask uses. You can just think of it
as Flask, that’s fine. It uses some HTML here, but it also in
the bottom of the file uses JavaScript. And so just as before when we’ve
looked at the source of the page, what I’m going to see in the browser
on this forms page is no jinja, no Python, nothing related to Flask. Just a fully formed HTML
page that also now contains some of that client side code. And so I have this mixture now
of several different language, but each of which solves
a very distinct problem. Yeah? AUDIENCE: So I think it was a
week ago or two weeks ago when we were working on
JavaScript and CSS, you were saying that it’s preferable
to split languages, not mix them in the same document. Right now we are mixing
multiple languages. DAVID MALAN: Really good observation. So a couple of weeks ago I really
preached the separation of concerns, and therefore separation of files. And that’s why we
introduced a .css file, we also briefly showed
examples of a .js file. The short answer is as your
applications get more complex, life starts to get messy. And the team and I were
actually talking about this earlier as to how to present
some of these examples, because what you are seeing in my
design decision here is a tension. So that tension here is as follows. It is not necessarily the best practice
to just have your logical JavaScript code comingled with your HTML. It makes it harder to
collaborate with someone else. If one of you is really good at design
and wants to work on the HTML and CSS, the other person really wants to do
the JavaScript code, kind of hard to do that when they’re
both in the same file. So we could factor this out. I could change this line, just
to be super clear, to be this. Instead of putting my actual code in the
file, I could do something like this. The source of this shall be
form.js, and that is just it. And then I have a separate file maybe
my colleague works in as follows. But at some point the thoughts
that go through my head are, it’s only like 10
lines of code and I just have to create a second file
now, and that second file is going to be maybe in a
different folder as my template. And you know, it feels like this
is just overengineering a solution to the problem. However, once it’s 20 lines, 100
lines, now OK, now it’s feeling messy. Somewhere there’s this inflection point. And this is where reasonable people will
disagree, and I might argue one way, you might argue the other way. And honestly, both of
us are probably right. And so this just speaks to the
web’s development over time. And there’s fancier frameworks
now. and if we tie things earlier into the question about CS50 beyond,
an opportunity after this class that looks more closely at
web programming, there are even fancier frameworks nowadays
than Flask and than Bootstrap that if tried to solve this problem. React is one of the most popular
ones developed by Facebook, now open source and
used by so many people around the world that
actually addresses this issue. And it allows you to separate
your HTML from your CSS from your JavaScript in different parts
of the file but still in the same file. And that was their particular solution. And View and Angular, there are so many
different solutions to these problems. And unfortunately, once we take
the training wheels of CS50 off, this is what’s ahead of you. The world is messy. And the reason there are so many
darn languages and frameworks is because people like you have
these instincts and think, this could be done better. And thus do we iterate and
have new and new technologies. But this is the kind of
stuff– and honestly, this is the kind of silliness
that changes over time. The fundamentals of HTTP,
and client side code, and JavaScript code, those
fundamentals are invariant even as the implementation
details change. So the short answer is,
could this be better? Probably. Could it be much better? I don’t know. It really now becomes more
of a debate among developers. Good question. All right, so let’s now use
some of these basic building blocks to make a final set
of examples that demonstrates a feature with which most of us are
pretty familiar reminiscent of what we did two weeks ago with Google Search. At the time we searched for cats. Today we’ll keep it a little
simpler and a little less graphical and just search for words. Because you’ll recall from
our speller problem set, you implemented a spell checker
with 140,000 plus English words. That’s a pretty juicy
dataset to search over, and you’re probably all familiar
with autocomplete these days. There’s hardly a website these
days that when you start typing it doesn’t try to finish
your thought for you. Google, Facebook, any a
number of other sites. So autocomplete, how does that work? Well, let me propose the
following mental model. If you do have some data set
like a big list of words, or a big list of Facebook friends,
or a big list of whatever, you might store that server side
because it’s a lot, a lot of data. And in fact, next week you
might store in a big database. But for today we’ll
just store it in a file like we did for the speller piece set. But if you want to create
an interactive experience for the human, what
language are you’re probably going to want to use so that he
or she gets immediate feedback? Probably JavaScript, right? That’s the whole principle. Client side code is just going
to execute faster because there’s no internet between you and the code. But with Python, you
have access to files. And yet with JavaScript code you
have closer access to the user, so there’s these tensions. So how could we go about building a
site that lets a human via form search across that file for words? Well, let’s start as follows. So in word 0 we have the following. Large, which is just a text file
borrowed from the speller problem set, 140,000 words, one poor line therein. I’m not even going to
double click and open it because it’s so darn big it’ll
take a few seconds to open. In application.py we have probably
the entry point to this application, and in templates we have just
three templates this time. So just when you’re reading someone
else’s code for the first time, where should our entry point be? Where should we start looking
to understand what’s going on? Maybe application.py. Or honestly, you know what? If you want to see what
something does, run it. No harm in doing that. So lets run Flask run. Make this a little bit bigger. Let me open up the URL here,
open, and I see a very simple form asking me for a query. Let me go ahead and search
for a and click Search. And after a moment, OK. This is a lot of words,
but apparently these are all the English words
that our dictionary knows about that start with the letter a. And if I go all the way
to the bottom, you’ll see it stops with az whatever
with no B words, in fact. Well, let’s make sure this actually
works and isn’t just a trick. Let’s search for b words. OK, so that seems to work, as well. And notice I borrowed some
inspiration from Google. Notice that the route I’m using is
called slash search like two weeks ago. Does take a cue parameter for query,
and b is whatever the human typed in. So if I want to search
for z the words, enter, I should hopefully get back now z words. So now unlike two weeks ago, we
can implement both the front end and the back end for a search engine. But our search engine’s
searching now just for words. So let’s look at application.py as
proposed, which is the entry point, and let’s see how I’m doing this. So this is some code that I
borrowed a little bit from last week when we quickly implemented
the spell checker in like 12 or 20 lines of Python code. I’m declaring a global
variable called words, and I capitalized it just to
be reminiscent of last time and the problem set. I’m using this syntax
which I alluded to earlier is just more conventional or Pythonic. Open the large file in Read
mode and call the variable file. Then here is a for loop via
which you can iterate over every line in the file,
reading one at a time. But recall, what does every
line in this file end with? Like a backslash n, and we don’t
really want those as part of the words. That’s not part of the English word. So R strip, right strip removes any
whitespace from the end of the string. And that’s why I needed
to add that extra line. So I’m just cleaning up the
file or massaging the data as you might do with any sort
of data based application. So then I just seem to have this
route that renders the template. If I look in index.html,
let’s follow the bread crumbs. Go into index.html. OK, not that much going on here. Looks like an HTML form, the
action of which is slash search, just like Google’s. The method of which is is
get just like Google’s. There’s nothing really private about
the words I’m searching for here, so I don’t care. There’s some fancier features here. Notice placeholder is the
grayed out text the human sees. Auto focus. What does this do again? This is just a UI feature,
better user experience. AUDIENCE: Puts the like,
right in the text box. DAVID MALAN: Yeah, it puts the
cursor right in the text box. To focus on something in a
web page means make it what’s interacting with the user right now. And Mac OS, for instance,
highlights it in blue. So when you first load
the page, the cursoe’s blinking in the choice of text
boxes that you care about most. Autocomplete off just disables the
browser’s version of autocomplete. So I don’t see past searches,
just because the whole point here is to implement this
ourselves ultimately, and then I have my search button. This is just jinja stuff from
Flask so that I have a layout file. Lets follow that breadcrumb. Lay out.html, nothing really
that interesting going on there. If you’ve ever wondered why we
have these in a lot of our demos, this cryptic looking line here
just makes web sites look better on mobile devices. Typically by default if you pull up
your phone and look at a website, if it doesn’t have
that kind of line, like the text is going to be super
tiny unless you pinch and zoom. By using this line and
variations thereof, it will increase the
default font size a bit to make it a little more
tolerable on small screen. So it’s an easy win
for users experience. OK, I seem to have exhausted all the
interesting stuff in these templates. Let’s look at another and final route. Here’s my search route, and
this is pretty Pythonic. This is a mouthful, and
will re-implement it in a different way in just a moment. So I have a search route that listens
for get requests on slash search. Then this crazy looking line is
about as Pythonic as code gets. And I’ll explain what
this is doing and why it’s conventional as opposed to
straightforward at first glance. And then I render the template,
passing in these words. So this one liner on line
17 actually has the effect of searching 140,000
words for whatever words start with what the user typed in. See, this would be a
pain in the neck to do. In Python you can do it
with literally one line. A long line, but one line nonetheless. Let me make this more clear. If I were to search for
words in this big file, I might do something like this. Words is an empty list. So this lower case words is
all of the words that match, that I want to send back to the user. So by default I have no
idea what to send back. But I do know I can do this. For word in Words– which is the capitalized variable,
the constant up at the top that has the whole–
or not even constant, but the global variable that has
all of the words from the file, here is a for loop over those. I can now say something like this. If the current word starts with
whatever the user typed in– well, what’s the user typing in? Well, q equals request.args.get
quote-unquote q, gives me the user’s name, dorm,
or in this case Q value for query. So if the word that we’re currently
iterating over starts with q, I can go ahead and append to my–
whoops, append to this list that word. Would you say you’re comfortable
with these lines here? To recap, give me an empty list in
which to store the search results, iterate over all possible
140,000 plus words, get– and actually, this was stupid. I should just put this up here, because
I only need to check for that once. So store the user’s input
in a variable called q. For each word among the 140,000, check
if it starts with the user’s input– a, b, z, whatever. And if so, append it to there. So let’s temporarily get rid of
this and just render the template. So notice this gets the job done,
but this is very C-like logic. It’s not wrong, it’s perfectly correct. But Python is a language
that’s meant to be a little more human readable
and a little more elegant, if a little more non-obvious. So this one line does
the exact same thing using a feature called a
list comprehension, which is ironic if you don’t quite
comprehend how it’s working. But here’s the variable I
want to create called Words. These square brackets
here say give me a list. What do you want to put in that list? I want to put a word in this list. Which word do you want
to put in this list? The result of inducing this loop
and then only putting in this list a word if it starts with
what the human typed in. So it takes some getting
used to, but this is just a one liner way, a very Pythonic way of
expressing those several lines of very procedural code into a simple one line. Is it better? Not if you can’t read it. But once you get more
comfortable with Python, yes, it’s better because it’s less code. Yeah? AUDIENCE: You said Python uses
notation to tell where conditions are. How is [INAUDIBLE]? DAVID MALAN: Good question. In this case of a list
comprehension, you can only have one line or one condition. You can’t have multiple lines therein,
so I cannot start hitting Enter and indenting here. It’s just not allowed. So you would only use this– and I’m
frankly really pushing the limits. You should only really use this
syntax when it fits on your screen or fits on a reasonable person’s screen. After that you should probably do
something a little more expressive. Other questions? But this is very common to see online. So any tutorials, if you ever
see this kind of one liner, just try to think about it from that
approach what it is actually doing. OK, so propose from a user
experience perspective how could this program be better? Because this is just our first version. So what could be better
for the user than this? What could be better? Yeah? AUDIENCE: Just going back,
can you explain words in caps? DAVID MALAN: Oh, sure. Words in caps is this global variable I
defined up here that stores all 140,000 plus words. That’s the really big file
called large, the text file. Down here I just need a local variable. And if it’s more clear, I could
call it results and then just say results equals results. That is the subset of words
that start with a, or b, or whatever the human typed in. That’s all. Good question. Yeah? AUDIENCE: Why do we
change the last time– why did you have to change args to form? DAVID MALAN: Why did I
have to change my args to– AUDIENCE: Forms? DAVID MALAN: So earlier today when I
didn’t understand what was going on, you should use request.args
for get requests. You should use request.form
for post requests. AUDIENCE: But it’s still .get after? DAVID MALAN: It’s always .get, yes. But you change what you’re
getting things from. In an ideal world, it would have been
something– oh, I see what you mean. Get in this sense is the verb. We humans mean go get something. Args in this sense,
if they had done get– I’m making this up, but this
is probably why they did this. Because get from get seems weird,
whereas get from post is less weird. But it’s just they called
it args and form instead. OK, so let’s actually
improve this, but how? What could be better for the user? Yeah? AUDIENCE: They could search a
word, not just the first letter. DAVID MALAN: OK, maybe
searching a whole word would be good, not
just the first letter. What else could we do? AUDIENCE: We can create an
index list of letters and words? DAVID MALAN: OK, we could
create an index list. So maybe using a hash or some form
of inspiration from our problem set with Speller and actually use
a more sophisticated data structure to get these answers more quickly. And let me propose, too, the goal here
is to actually implement autocomplete, and this was not auto complete. This was like old school search. Type in a query, hit Enter,
get a page of results. What if we want to do
something more immediate? So let me actually propose this. Before looking at the code,
let me go into words 1. Let me go ahead and run
Flask in that directory. Let me go ahead and
reload the form here, and now notice no Submit button
because there’s not going to be any actual submissions here. But I’m going to go ahead and hit the
letter A, and ooh, that’s kind of cool. Let me delete that, goes away. B, there’s all the B words. Let me go B-A words, B-A-B
words, B-A-B-A words. This is how autocomplete works. So it seems to be responding immediately
to my input, so something’s happening. But I’m not actually
submitting the form. So I’m kind of using it now,
it seems client side JavaScript to maybe talk to the server? Let’s infer. So here, too, this
should be your instinct. Whenever you’re trying to understand
how someone’s website works, if you want to learn from it or mimic
certain fundamental functionality, go ahead and inspect the page. And you don’t probably care
too much about the HTML yet. Where is this data coming from? Let me click on the Network tab, which
we looked at a couple of weeks ago. Let me go ahead and restart this
and let me clear this and start from the beginning of the story. Let’s see what happens when I
type the letter A. Interesting. There is a web request. So if I zoom in down here, notice
that my browser actually searched for Q equals A, the human’s input. Let me go ahead and– it
keeps searching because I’m using keyboard shortcuts here. But let me go ahead and click this row. Notice what happened. I made a request to slash search
question mark q equals a via get. Let’s see what the response was. The response here, if
I view the source– or rather, if I read the response– notice what came back. It looks like my server returned to me
a fragment of HTML containing hundreds, maybe thousands of words starting
with A. But notice there’s no UL tag, there’s no head tag, no title, no body,
it’s just a partial HTML fragment. But that’s interesting, because I know
with Python I can do exactly that. I can generate anything
I want on the server, and then maybe the browser can just
plug in those changed the results. So let me go ahead and look
at the code for this page. If I go now to the browser’s
source code, the view page source, you’ll see a few new lines. So to do this easily, I’m
actually using another library. This one is called jQuery. This was for many years
super, super popular. It’s kind of starting
to fall out of vogue, but it’s still so
powerful and so useful. And it’s used by Bootstrap, the
other CSS library we’ve talked about, so it’s perfectly
reasonable to use it here. Notice how I’m including
it with the script tag, and it’s hosted on a
third party website so that I don’t have to save a
copy of it myself on my own IDE. Then let’s look at the
code I actually wrote. So notice that atop this file
is not even a full fledged form, it is just the HTML input. Because I don’t need a full form. I don’t need an action,
I don’t need a method, because I’m not submitting it
anywhere with the human’s cooperation. I’m going to use my own code. So in my script tag here, my
JavaScript code, notice what I’m doing. This is some code from
like two weeks ago. I’m going to search the tree
that represents this web page. And indeed, it is meant to be a tree. Recall from that time when
we looked at an HTML page, there is in memory, thanks to
the browser, something treelike– a DOM, document object model–
that represents your page. Using JavaScript, can we change
that page after the fact? So what am I going to do? I’m going to tell the browser whenever
this input hears an event called on key up– so whenever the field has
focus– it’s blue in Mac OS– and the human hits the
key and then let’s go, and the key goes up, go ahead and
call the following anonymous function. What do you want that to do? Now, this code is a
little cryptic, but let me walk us through it because
it’s only three lines. This code here is using
a special feature– dollar sign– that comes from
this library called jQuery. More on that in a moment. That library, somewhat confusingly
named, has a function called get, which has nothing to do with Python
or the one we just talked about. But this has to do with an HTTP get. With this line of code,
you can tell a browser, even after a web page has been loaded,
go get me this other URL, please. So what URL do you want to get? Go ahead and get me from the same
server slash search q equals, and then what does plus mean
in JavaScript if you recall? Concatenation. So it means just append
one string to the other. So this is like saying,
go ahead and get me the URL that ends with slash search,
question mark, Q equals A, or Q equals B, or Q equals Z. Whatever
the human typed and just gets slapped onto the end. And then that’s where
we’re getting it from. Input.value is the user’s
input, the value thereof. And then the last line– and
this is perhaps the fanciest– notice that I have an
anonymous function. In this library called
jQuery, there is this function called get that gets a URL. When the server responds to your request
with a virtual envelope of its own, this anonymous function gets
called and the response envelope gets handed to you, so to speak, as
a data argument, as a data variable. Then what you can you do? Document.queryselector UL. What is UL? It’s an unordered list that by default
on this page has nothing in it. But recall that what the server is
sending back is a bunch of LI tags. That’s great, because I want to put
those LI tags right in between here. So how do I do that? I go into the so-called
inner HTML of the UL tag, and you might not have seen this before. But you can change the contents
of an existing tag inside of it by using inner HTML and
just plop the data in there. And so what’s happening is this. Let me go ahead and open
up Chrome’s inspector. Reload the page so it’s empty. Let me open up Chrome’s inspector. Go to elements, as is the
default. And notice on this page, notice that UL tag is opened and
closed with nothing inside of it. The moment, though, I search for
something, watch what happens. If I search for a, all of a sudden– ooh, it blinked, it’s a little small. Now there’s a little triangle there. What’s inside of it? All of those LI tags that
came from the server. So with JavaScript, we
have this amazing power now to change what’s inside of a web page
by just asking the server for more data. So if you’ve ever used
Facebook, or you’ve used Google Chat, or any websites that’s
dynamically changing every second, every minute and each time you
get a message, you can literally, if you get a little nosy, open up
Chrome’s inspector and watch the DOM, watch this elements tab. And you’ll see new stuff
popping up every time you get a message, or a chat, or any
other such notification on the screen. Now as an aside, this is a little
sloppy to be returning HTML, but let’s see how it’s done. Let me go into application.py for
words one, which is this example here. And in application.py, notice
what I’m doing is this. Rather than return a
whole page of results, I’m returning a template
called search.html. All of the rest of this
code is identical to before. If I go into my templates and go into
search.html, look how terribly simple the code is on the server. If all you want to do is spit
out a bunch of list items, this is all you need. There’s no template. Like there’s no extends layout because
you’re not returning a whole web page, you’re returning a tiny,
tiny, tiny fragment of HTML. But this is arguably a little
sloppy, because there’s a lot of redundancy
in what’s coming back. If I look at this tag that’s
coming back from the server, what is obviously redundant about all
of this information that’s coming back? And if I look at the Network tab,
you really see it under response. What’s redundant? AUDIENCE: You’re doing a bunch of calls
to the same address and to the same– DAVID MALAN: This was just because I
hit some, like, zoom in and zoom out, so it pretended to
make multiple requests. So red herring there. Focus only on this part here. What’s redundant about all
of the data coming back? It’s just keeps saying list
item, word, close list item. List item, word, clothes list item. I mean come on, just use
a more efficient syntax. Just separate things with commas
or something lighter weight. This is sending way many bytes. I mean, look. There’s thousands of bytes. This is kilobytes by
definition of information that we’re sending just
to send open bracket, LI close bracket again and again. This is not very efficient. And so the world actually has
adopted a different approach, and I’m going to show this in words 2
that actually returns something called JavaScript Object Notation, which is
a more succinct representation of this as follows. Let me go into words
2, run Flask in there. Search for the same kind
of thing and then watch what happens over the
network panel this time. When I search for A, immediately
get back the same visual result. But if I look at this search
query, now look what comes back. I claim that this is a much more compact
representation of the same information. It’s a little annoying
that there’s double quotes, because those are a little redundant. But at least double quotes
are a lot more efficient than open bracket, LI, closed bracket,
and then the opposite at the end. So this is what’s called
JavaScript Object Notation. And as this square
bracket here and thousands of words later the square
bracket on the end implies, this is a JavaScript array that’s
being sent back from the server. So the only thing that’s
changed here is as follows. In words 2, this example,
notice that I don’t even need to return a template anymore. This code is the same as
the past two examples. This is how I’m searching
140,000 words quickly. But if I include now a fancier
function from Flask called jsonify– which is not really a word. But jsonify, that takes
any data structure you have data in like this
list of words, the matches, and it turns it into that text based
representation with quotes and commas. And you don’t even have to
write a template yourself. And indeed, I got rid of search.html. The only thing you have to do to
give yourself access to this feature is import not just render
template, and request, and Flask, but jsonify, as
well, from the Flask library. Which is just one more feature. Any questions on that before we bring
it all together with one final example? Yeah? AUDIENCE: Can double quotes
break that into [INAUDIBLE]?? DAVID MALAN: Can double
quotes break that? Good question, great instincts. No because the author of
jsonify was smart about this. And if that author notices a quote
like an apostrophe, or like something in your own name that has a quote,
it will escape it in some way, usually with a backslash. Good instincts. But that’s why you probably wouldn’t
want to write this code yourself because then you have to think of
all of those corner cases as opposed to focusing on the parts you care about. All right, so there’s one final example. And it’s perhaps to come full circle
here, do we even need the server? These 140,000 words right now
are in a file called large. My web application loads that file
into memory and then searches it. But who else could search
over a big list of files? Where else could we put this logic? The browser, right? Browser gives you JavaScript,
JavaScript’s a language, languages can search things. So let me try this instead. In our words 3 example here,
notice that I’ve got one new file. In advance, you know, I took my big text
file that just had one word per line and I put it into a standard
format just because it makes my life a little easier. And I called it large.json. And in there, actually, where if I open
up this folder, you’ll see large.js, which is a second file,
this time a JavaScript file, in which I’ve just declared a
JavaScript array of all 140,000 words for better or for worse. I just put them into a
slightly different format with commas, and quotes,
and square brackets, and I gave this whole thing a variable
name at the very top of the file. Now, why is this useful? Well, if I go into
index.html, notice that there is no more application.py or
templates for this example whatsoever. We’ve gotten rid of
Python entirely, I don’t know if it’s for the better
or worse, but we’ll now see. So in this file, notice we
have an input tag as before, a placeholder for all of
the ULs, but we’re also now including this large.js file. Thereby telling the browser
please download all 140,000 words and then search them locally. How am I going to search it locally? I’ve essentially converted the
language I wrote in Python a bit ago into JavaScript as follows. Here says the browser go get me the
input that the user can type into. This tells the browser, go
ahead and listen for key up. Whenever that happens,
please call this function which has no name, because I want
you to just call it immediately, I don’t need its name. This function is defined
by these several lines. These several lines have to do
a bit more work than before. Because before the server
was doing all the hard work sending back all of the data, and
we just jammed it into the web page. But here I’m going to build up
a list a little more manually. So I’m going to let a
variable called html equals quote unquote, because I want
to build up the unordered list myself. Then if the human, indeed,
types something in– so if their input is non null, so
if they type at least one character, do the following. This is weird in
JavaScript, but when you iterate over an array in JavaScript,
you use the preposition of not in. So for word of words, go
ahead and do the following. If the current word starts with–
notice the capitalized w– also different from Python, but same
idea, just different spelling– if the word I’m iterating over starts
with whatever the human inputted, we found a match. Go ahead and append
to this HTML variable, open bracket LI closed bracket,
concatenate the word to it, and then close bracket, as well. So I’m constructing a variable
in the browser’s memory containing HTML that I want to
jam into the DOM ultimately. How do I do that. Well, at the very last line I say
to the document, select the UL tag, go inside of its HTML,
and change whatever is there to this string of HTML,
which is presumably 0 or more LI tags now based on those search results. So now let me go back to words 1– rather, let me go back
to this example here. Let me go ahead and serve this up. It’s not Flask anymore, so I have to
use our server from two weeks ago, HTTP server, to serve up HTML. Let me go ahead and reload the screen
here, open up index.html, and now notice we’re good to go. What do you want to search for? A, B, C. And let’s open
up the Network tab. Inspect, Network. Let’s see what happens
every time I search. Z, Y, Q, A. Why is there
no network traffic now? AUDIENCE: There’s no network traffic. DAVID MALAN: OK, but that’s what I said. But why is there no network traffic? It’s not sending any routes, it’s
not talking to a backend server. Why? Because all the data I
might need is already local. So mixed messages here, too. Which is better, which is right? What’s the takeaway? How do you think about this? Because now, whereas a lot of our
programs early on in the semester were relatively small even though
they didn’t feel that way at the time, now we have even more
design possibilities. And the answers are
increasingly non-obvious, and this is why you as
programmers will just get more comfortable with
conventions, you’ll maybe practice what you’ve
seen preached first, then you’ll decide as you
might be already saying, I don’t like that, I’m going
to do this some other way. So how do you think about which of
these several words examples is best? Version 0 was let the
server do all the work and just send back a full new page
of results like Google in 1999 did. Then version 1, we added
a bit of JavaScript that used jQuery the library
to talk to the server using a technique called Ajax– asynchronous JavaScript and XML– but that just means go get me more
data and we returned LI elements. Then we decided that’s
a little sloppy, don’t seem to send me all
these useless tags, just send a comma separated list of words. That was version 2. And then this last version
gets rid of all of that and just sends all the
words to the browser and lets it deal with it entirely. Who likes which best, 0, 1, 2, or 3? What do you think? AUDIENCE: So if you aren’t
supposed to trust the user in case they turn off the JavaScript, then
their decision of the functionality of the website works. DAVID MALAN: That’s a good trade-off. So if the user turns off JavaScript, 3– OK, it won’t offend anyone– 3 2, and 1 won’t work anymore
because JavaScript’s disabled. Now, do you care about that? Maybe not. It’s a small number of
users on the internet who are so concerned with this they
just turn off JavaScript altogether, and the reality is so many websites just
break these days without JavaScript. That might be a reasonable
cost of doing business. Other thoughts? Yeah? AUDIENCE: Is the last
version the fastest? DAVID MALAN: Is the last
version the fastest? I don’t know, how could we assess that? Yeah, I mean, we can
literally measure it. And honestly, built into Chrome in
other tabs that I’ve not even clicked on are a lot of performance
tools where you can actually monitor how long everything’s taking. This is called benchmarking
more generally, and this is what you did
essentially in the speller problems that even though we wrote the
code that timed everything and measured everything,
to answer that question, you could just try timing
every one of the examples and then decide for yourself
which is best, sure. Yeah? AUDIENCE: The last thing I
have with problematic issue is like my phone’s old, and it
doesn’t have a lot of memory, and I may not have the
fastest plan on Earth. So you might be charging me more data,
you might be slowing my phone down. DAVID MALAN: Very reasonable concern. Let me go into our dictionary file
here and list the size of this. That large.js file is 2.2
megabytes, which on mobile devices, especially in places where your signal
is slow, or your bytes are expensive, or you only have
intermittent access, I mean, that’s kind of obnoxious to
send two megabytes to the user if you can avoid it, especially when
you can send only the subset of results, a few kilobytes maybe,
of matches instead. So a very reasonable concern. Cost, user experience, performance. Other thoughts? Which is better for people on university
and corporate campuses who tend to have really good internet access? Say again? 3, OK. But what if it’s not
140,000 words, but it’s like a billion pictures of
cats that Google indexes? Or hundreds or thousands of friends
and all of their profile data? Like at some point there’s
this inflection point where too much data, right? It’s not reasonable, it’s too much. You don’t want to just get a
copy of your corporate database to every one of your users just
for them to search more locally. So again, these are the
non-obvious questions. And one of the goals of,
frankly, the final project if you do something web based, or
mobile based, or C based, or anything, is all of these questions
now boil down to you. Like we have no
well-defined answers in mind as to what your final project should
do and how it should do it best. But in the weeks ahead, you’ll be
pre-proposing some project ideas, proposing an actual project idea, and
then designing and implementing this. And you, too, will
probably feel this tension where the answer is not always obvious,
your teaching fellow might not even know how to answer your
question, because he or she will have thoughts of their own. But it’s ultimately up to you. And we’re now at the point
of a level of coding maturity where we’re taking the training
wheels off and more of the decisions are now left to you. Let’s end here. Stick around for one on
one questions, and we’ll see you next week for databases.


Leave a Reply

Your email address will not be published. Required fields are marked *