Django

Django Dynamic ModelForm Using Closure by Eric on February 1, 2020

At my job we are currently undergoing a total rewrite of their old web-based data entry system.  It's written in Django and works pretty-alright we are just going to make it better.  When it was first written the company was small and very few tests were being done.  This caused the initial design to lack foresight into the growth of the project.  Another problem with this current system is that it was an almost direct port of their old Microsoft Access version they had been running before.  The company is international and a web based system is much better and easier for the employees to use but wasn't designed with growth in mind. With a better understanding of how the systemis being used and what the users wanted it to do we have been in the beginning phases of development for about three weeks now.  Parts of the site are ready to go live within a week but the major parts are left. One obstacle that was brought up by one of the software engineers today was the idea of creating ...

Django works wonderfully as a web framework but sufferes from sub-par data migration.  Luckily for all of us one of their developers has created an application called South that fixes this very problem.  South is meant to bring data migrations to Django and make it easy to make schema changes that are quickly applied to your database without much effort on your part.  Django does have a few built in features for dropping tables, creating tables, and will give you the code to enter to create the tables but it doesn't help much when you are working on a production server that you don't want to simply drop and readd tables all the time. To get started just go through the installation tutorial.  Once you've done that you're ready to migrate your data!  This is best done from the very creation of the project so South can help you from the beginning.  If you are starting a new project go here and follow along.  I'm going to focus a bit more on converting an already created project and ma...

Get List from POST Data - Django by Eric on February 1, 2020

Recently I tried sending a list through a form using something like <input name="my_list[]" value="some val">.  When this form is submitted all of the inputs with that name are grouped together for easy access.  In Django, one would think you could simply access the list using request.POST['my_list[]'].  This is not the case. Using that command will give you the last value.  Django says this is a feature and to get the list use request.POST.getlist('my_list[]'). Just an FYI and reminder to myself.

Adding Custom Django 404 and 500 Pages by Eric on February 1, 2020

Django comes with a default 404 and 500 error pages which are dull and boring.  It would be much better if you add your own spin on those pages to inform your user of what happened.  Django makes this super easy by searching your project template files for a 404.html or 500.html file when it tries to render those pages.  This means all you need to do is make a nice Django template called 404.html and 500.html and place it in your project templates folder and you're done!  You'll have access to whatever context processors you are running for information, most of the time these pages won't be user specific and will just display some error. Please note that you won't be able to see them unless you turn DEBUG = False in your settings.py file.  Once debug is off Django will render the 404 and 500 pages rather than the debug output. Hope this helps people get their pages a little more user friendly and comfortable.

Django Context Processors by Eric on February 1, 2020

Today I started work on allowing users to edit certain settings we have for them, namely what office they are in and their language.  Using Bootstrap I changed the way our navbar worked so clicking on your name drops down to give you the option of editing your profile or logging out.  Editing your profile brings up a modal with the choices you can change.  This modal is located in our project's base.html file so that users can change their settings on any page.  Generating the form with the options was where this post came in. For a while now I've been using Django's default language changing form.  In it they reference a variable LANGUAGES.  For the life of me I had no idea where that variable was being passed in from.  Now I needed my own variable, OFFICES, to be passed into every page as well.  After a quick Google search I found out that these variables are passed to the templates through context processors.  In your Django settings file you can see all of the context processor...

Django 1.6 Update Notes by Eric on February 1, 2020

Django finally released version 1.6 of their Python based web-framework.  I use it at my job everyday and we ran into issues with the old system where they were using such an old version of Django that it was impossible to just simply update.  One of the goals with the new system we are creating is to stay as up to date as we possibly can with all of our technologies.  For us that is Bootstrap, jQuery, Django, and a few other third-party web plugins.  Here are a few of the notes I'm writing down for myself and for others to use if they run into issues in upgrading from 1.5.4 to 1.6.Default session serialization switched to JSON Django changed the way the session variable is being stored from pickle to JSON.  This is for the best as there is an exploit if the attacker knows your SECRET_KEY and has access to the CSRF tokens being sent by users and the session variable uses pickle.  Using JSON fixes this and Django has elected to make it the new default.  The issue is that some things...

Django Messages Framework and Custom Tags by Eric on February 1, 2020

![null](/img/screenshot-from-2013-08-22-090436-300x144.png) Sometimes it's necessary to display a message to the user in a quiet and clean manner.  To do this, Django has given us the messages framework for cookie- and session-based messaging.  After a form or other input is submitted we can quickly send a short message to the user without much effort at all.  Here are all the documents related to the framework but I'm going to give you the quick version as well as some customization options and Bootstrap fixes. The messaging framework is enabled by default when you create a new Django project so good job you're half way there!  Now just add this bit of code to the top of any page you want messages to appear:{% if messages %} <ul> {% for message in messages %} <li{% if message.tags %}{% endif %}>{{ message }}</li> {% endfor %} </ul> {% endif %} We have this in our base that is inherited by every page to allow the functionality to appear on the entire site.  You need t...

MySQL Queries for Grouping Data By Year by Eric on February 1, 2020

A big request we had in designing our new system was an easier way for users to get data from the database.  With the large amounts of data that we use across dozens of tables it can be slow going to use the Django methods.  Instead they have given the ability to use and write raw SQL and use that result how you please.  You can read about using cursors here, I may do a post in the future about them but today I want to focus on the SQL we wrote and the problems we encountered. The requested query was to get a list of customers, how much business we did with them, and break that up by year.  The year could be inputted by the user so it needed to be dynamic enough to scale.  The first way we tried was to create a temporary table for every year needed.  Each table was created from a SELECT statement that was specified for the office as well as the year.  Below is the SQL command for one of the offices and years but you can imagine we copied this code, almost verbatim, but changed the ...

It's been far too long since my last post.  A lot has happened and the holidays make things so busy but I'm back and ready to get back in to the swing of things.  To start off the new year I'm going to do a series of posts relating to ReportLab specifically with its application to Django.  I want to go over a few of the pitfalls I've experienced and how to get things set up.  ReportLab is notoriously tricky to get working and the documentation seems sparse to say the least.  I hope these posts help not only myself remeber how I did all of this but others who are struggling with the same problems.ReportLab The problem is simple, how to generate PDFs using Django.  In our on going rewrite of our system at work we are getting to PDF generation.  The old system uses wkhtmltopdf which is great if you just want something to render HTML out to a PDF and you don't give much care to speed or formatting.  Wkhtmltopdf suffers from having to use HTML to format everything on the page.  This is l...

Django Translation Creating Locale Folders by Eric on February 1, 2020

As our huge project progresses we need to start thinking about some of the major features that our users would like to see.  The first is less of a feature and more of a necessity.  Our users are located all over the world and speak four different languages.  To accommodate all of these users we need to translate every bit of text on every page of our application for all of our users.  Luckily for us Django handles translation very nicely.  Read up on the Django translation here. This post will only cover the template tag {% trans %}, more will come as we develop more of our translation setup. Every template generated will need the tag {% load i18n %} at the top to have access to the trans tag.  From this point on any text string will need to be surrounded by the trans tag.  For example, my button has the code `Home` and to use the translation I simply change it to `{% trans "Home" %}`.  That string after the trans is going to be the id for the soon-to-be-created translation files...

Django Timezone Conversions by Eric on February 1, 2020

Django does a great job at converting datetime objects to whatever the servers timezone is when your settings are set to do so.  For our big project we needed times to be local to the user.  Django documentation says to allow the user to select their time zone and save it to their user profile, which we plan to do.  Until then, to get things working, a few tests were in order. I have a page where a user can make a note about a company and a timestamp is created and displayed for when the note was made.  When the template is rendered it takes the UTC specific time saved on the database and converts it to our local server timezone of 'America/Denver' automatically.  To make the site more responsive, and less hits to the server, I make the creation of these notes happen through an AJAX call.  The information is then displayed on the list quickly and quietly.  The problem I was running into was that the JSON object returned in the AJAX call had the UTC time and not the local time for t...

Add a Simple Google Map to Your Webpage by Eric on February 1, 2020

![null](/img/map-title-300x164.png) There's just something about having a map of an address that is appealing to people.  Why just list the address like a mailing label when you can place a nice marker on an interactive Google Map?  That's what I thought too when I was adding content to our customer profiles.  I'd never done this before but knew that Google has a great API for accessing their data as well as some pretty good examples to follow.  The API covers how to use Google Maps on any device but for the purposes of this post I will be focusing solely on web. Let's go over some ground rules for using Google Maps API and then move on to a very simple example and build upon it slightly.  To start off with, you can only make a limited number of requests to Google for maps data before you have to start paying.  For a small company or webpage this limit will not be a problem.“For-profit web sites are permitted to generate up to 25 000 map loads per day using the Google Maps JavaScr...

Starting Again by Eric on February 1, 2020

Hello, Welcome to my new blog. Created for me to take notes on my every day workings and findings in the areas of web development and computer science. I will try to update regularly with notes that I take each day working on the various projects that I do. A little about myself, I am a recent graduate of the University of Utah with a Bachelors Degree in Computer Science. I work full-time as the software engineering manager for a textile testing company and do sub-contract work on the side. I created this blog as a place to keep my notes, establish discussion, and learn the tricks of programming and designing WordPress sites. The site is named Eric Saupe, as that is my name, and the tagline (for now) is Things my wife doesn’t understand. Often while I was at school and coming home from work I would want to tell someone about the cool new things I learned or discovered and my wife, Carly, was always the only one around so I would tell her. She would smile and nod her head bu...

Django Security Advisory: ImageField Abuse by Eric on February 1, 2020

A couple of weeks ago it came out that there is a flaw in Django's ImageField which could potentially allow for phising programs to be uploaded and grab cookies or do other malicious things.  While there will be no fix in Django directly you still need to take precautions on how you serve and receive files uploaded by your users. Django has a page dedicated to fixing this exact issue.  Head over to their security guide and read up on the fixes.  They shouldn't be too hard and shouldn't affect any user experience. Just wanted to post and let everyone know of the vulnerability.

Last time we looked at how to generate a very simple PDF using ReportLab and Django, ReportLab and Django – Part 1 – The Set Up and a Basic Example.  This time let's make the PDF a little bit more interesting with some headers and footers.  ReportLab gives a pretty good amount of control when it comes to adding headers and footers to your PDF.  To start off let's bring back the simple __init__ method we had from last time and add a header/footer method to it.from reportlab.lib.pagesizes import letter, A4 from reportlab.lib.styles import getSampleStyleSheet class MyPrint: def __init__(self, buffer, pagesize): self.buffer = buffer if pagesize == 'A4': self.pagesize = A4 elif pagesize == 'Letter': self.pagesize = letter self.width, self.height = self.pagesize @staticmethod def _header_footer(canvas, doc): # Save the state of our canvas so we can draw on it canvas.saveState() styles = getSample...

Custom Django Admin Filter with Foreign Keys by Eric on February 1, 2020

During our massive redesign of our internal tools I wanted to make the Django admin page more friendly for us, the software engineers and admins.  Currently when they have problems or need things changed the easiest way to do that is through raw SQL and altering the database directly.  This is inconvenient and can lead to bad data.  Django, by design, abstracts that interaction and creates a great user-interface to interact with including filters and search options. One of the strengths of this is you can define your own custom filters that will appear as normal filters on the UI.  Defining them is quite simple and you can check out the tutorial directly from Django(Search for DecadeBornListFilter).  I wanted to take the query a step further and have it look up items based on a characteristic of a parent object stored as a foreign key.  Using the double underscore (__) Django allows you to access data from that foriegn key in queries.  Following the tutorial I was able to quickyl m...

MySQL Collation Matters by Eric on February 1, 2020

When creating databases and tables with MySQL there seems to be one setting that gets overlooked, collation and character set.  For most smaller sites, and maybe those purely in English, this may not be such a huge deal but for a larger application site and one that will be holding characters from various languages this is essential. UTF-8 is the collation and character set we want and the MySQL command to create a database with these settings is CREATE DATABASE dbname CHARACTER SET utf8 COLLATE utf8_general_ci; Bringing this back to Django, when Django creates new tables within a database it will use whatever the default character set and collation is for that database.  This is why it is important to get this right at the beginning with the command above.  If you do not, however, there are ways to make future tables created have the right encoding.  Simply add these lines at the bottom of your database connection settings in your settings.py file: 'OPTIONS': { 'init_command':'SE...

Django Translation - Going Live! by Eric on February 1, 2020

A while back I wrote a post about Django translation and getting your websites to be translated.  Since then it has been pretty successful in our development version of the site and I have continued to learn the quirks of Django translation.  A couple of things to note: 2. In your .po files you will occasionally see that some of your strings have a #, fuzzy tag above them.  This means that there is already a translation found elsewhere and it's not sure if it should use your translation file in this app or the other one it knows about.  I had a lot of issues with this so I have been removing the fuzzy tag entirely when I find it and this forces Django to use the translation file as is. 4. Ordering matters in INSTALLED_APPS.  Django will load translation files in order of your INSTALLED_APPS list.  Django has a set of built in translated strings that it will use first before using your translations.  I found this was causing an issue when I didn't like the translation used for the...

Django DRY URL's in Templates by Eric on February 1, 2020

![](/img/urls-300x164.png) With so many templates in a large project that all link together in creative and complicated ways it can be a pain to keep track of URLs.  Every URL is an address that can also have a number of variables associated with it.  If these need to be changed it can be very tedious to go through every file and replace the hard-coded URLs to reflect the change.  Helping with this, Django has a template feature that will look those URLs up for you and insert the relative URL, without the domain name, into your templates on render. It's a waste to have hard-coded URLs in your templates and if you are adhering to the DRY principle, don't repeat yourself, you'll not want to have the same hard-coded URL in many places in the same project.  Using the {% url %} tag you can reference a view and no matter what its URL is Django will handle it for you. Example time:urls.pyurlpatterns = patterns('marketing.views', url(r'^$', 'index'), url(r'^overview/$', 'overview...

Custom Column Widths in Bootstrap Tables by Eric on February 1, 2020

One of the biggest things I've run into with Bootstrap are their tables which are dynamic and grow and shrink depending on content size.  For the most part this is completely desired.  Every now and then you want a bit more control over your table column widths.  This is a very simple trick that I wanted to document for those who have run into the problem of wanting more control than Bootstrap gives initially. Say you've created your table as shown below.<table class="table table-condensed table-striped"> <thead> <tr> <th>Number</th> <th>Standard</th> <th>Category</th> <th>Labs</th> <th>Description</th> <th>Min. Sample Size</th> </tr> </thead> <tbody> {% for test in tests %} <tr> <td>{{ test.number }}</td> <td>{{ test.name }}</td> <td>{{ test.category }}</td> <td>{{ test.labs }}</td> ...

jQuery Drag and Drop Image Upload by Eric on February 1, 2020

As we continue to push forward on development our users have asked for a robust way of dragging and dropping images and linking them to a work order.  I was putting this off for a while because I thought it would be a pretty difficult task but after a few Google searches I found a great jQuery plugin from blueimp called jQuery-File-Upload.  Take a look at their demo and check out how awesome it is.  We didn't need any of the fancy bells and whistles that came with the Basic Plus UI so this tutorial focuses on just the Basic but you can scale yours how you'd like.  The tutorial will also focus on getting it to work and save to a MySQL database through Django. To start off let's get the drag and drop area set up on our page.  To do that you'll need to add these things to your HTML template, Django tags are added because that is what we are using to render the page.image_upload.html{% block css %} <!-- jQuery Upload Files --> <link rel="stylesheet" href="{% static 'jQuery-File...