You have a Django app running on a webserver and hopefully you're writing your logs to a file. If anything goes wrong you can search back through the logs and figure out what happened.
The problem is that to get to your logs, you have to log into your server, find the right file and search through the text on the command line. It's possible to do but it's kind of a pain. Isn't there an easier way to view your Django app's logs? Wouldn't it be nice to search through them on a website?
This post will show you how to push your Django logs into Papertrail. Papertrail is a free web-based log aggregator that is reasonably simple to set up. It stores ~6 days of searchable logs. It's best for small, simple projects where you don't want to do anything complicated.
Create an account
Your logs will go to logs2.papertrailapp.com:41234 and appear in Events.
You need to note down two things from this page:
- The hostname: logs2.papertrailapp.com
- The port: 41234
These two peices of information will determine where Papertrail stores your logs, and they're essentially secrets that should be kept out of public view. Keep the page open, because it'll be useful later.
Install Papertrail's remote_syslog2
Papertrail uses some tool they've built called
remote_syslog2 to ship logs from your server into their storage. Assuming you're running Ubuntu or Debian, you can download the .deb installation file for remote_syslog2 from GitHub. As of the writing of this post, this is the latest release deb file.
# Download installation file to /tmp/ DEB_URL=" https://github.com/papertrail/...deb" curl --location --silent $DEB_URL -o /tmp/remote_syslog.deb # Install remote_syslog2 from the file sudo dpkg -i /tmp/remote_syslog.deb
You can read more about remote_syslog here.
Create logging config
You can configure what logs get sent to Papertrail using a config file. This uses the YAML format and should live at
# /etc/log_files.yml files: - /tmp/papertrail-test.log destination: host: logs2.papertrailapp.com port: 41234 protocol: tls
Run Papertrail with a test log file
Start by testing out whether remote_syslog is setup correctly by running it in non-daemonized mode:
remote_syslog -D --hostname myapp
Note that "hostname" can be whatever name you want. You should see some console output like this:
... Connecting to logs2.papertrailapp.com:41234 over tls ... Cannot forward /tmp/papertrail-test.log, it may not exist
Make sure you have this page open in your web browser (or open it now). In another bash terminal, write some text to papertrail-test.log:
echo "[$(date)] Test logline" >> /tmp/papertrail-test.log
Now you should see, in your remote_syslog terminal, a new message:
... Forwarding file: /tmp/papertrail-test.log
When you look at the page you have open, you should see something like:
Logs received from myapp
Run Papertrail with real log files
Now that you're happy that Papertrail is able to upload log messages, you can set it up to ship your log files. In this example, I'm going to upload data from the Django and gunicorn log files I created in this post:
# /etc/log_files.yml files: - /var/log/django.log - /var/log/gunicorn/access.log - /var/log/gunicorn/error.log destination: host: logs2.papertrailapp.com port: 41234 protocol: tls
When you are not testing with remote_syslog, you want to run it in daemonized mode:
sudo remote_syslog --hostname myapp
You can check that it's still running by looking up its process:
ps aux | grep remote_syslog
If you need to stop it:
That's it! Now you have remote_syslog running on your server, shipping log data off to Papertrail.
If you have any feedback or questions email me at [email protected]