Working for Equity as a Software Developer

As a software developer I’ve been involved in a number of equity conversations over my career. Mostly what you’ll run into is dreamers with no experience asking you to work for free while they waste your time. Occasionally you’ll encounter a real business person who is worth talking to. It is thrilling to negotiate for equity – simultaneously weighing the adventure, challenge, and potential reward of a new job. Beware, it is a literal “shark tank” in this area of business and you’ll need a lot more than this blog post to guide you!

Below I’ll share some of my perspectives as the “tech person” being recruited into a startup with equity as part of the compensation package.

Disclaimer – I am not a lawyer and this is not legal advice. The information in this post comes from personal experiences and those shared by my colleagues. Examples provided have been anonymized and generalized into a work of fiction.

What is Equity:

Impulse Power vs Warp Power and the allure of equity:

Selling your time is like traveling at impulse power, great for short trips around the solar system.

Owning a substantial chunk of equity is like having warp power. Only warp power can take you to other star systems you can only dream of (eg, getting really rich).

I was once told “Equity is a funny thing.” and I agree with that.

Equity is ownership in a company. It has “value” but it acts more like red matter from Star Trek than regular money. Your job will be to help grow that value. With equity you can’t easily tap into it, transfer it, or even get rid of it. That is until an exit event where the company gets sold and all owners tap into the value at once.

Equity can also be valuable in a different way if you are the majority owner. Majority ownership provides control which gives you access to the company’s cash flow, hiring and firing decisions, and total freedom. Being in the position of a majority owner generally requires you to start the business and put in your own money. This is much riskier up front, but the potential rewards are much higher too.

 

How much equity can you expect?

There are Founders and Later Hires:

If you are coming in at the very beginning of the company you can get a lot of equity and possibly control but don’t expect to get paid right away.

If you are coming in as a later hire you can get a decent wage and a little equity (anywhere from a few percent to a fraction of a percent). Typically that equity will vest over a number of years (you have to stick around to earn it).

Working for a ‘Minority’ share:

A minority share is ownership that doesn’t offer control, typically anything less than 50%. When you are working for someone else who is funding the project, you will be a minority owner.

Working for free?

If you are going to build it for free, ask for at least 51% equity, or 100% of the voting stock and 50% of the common stock.

Equity changes with time:

If you don’t have control, expect to get diluted along the way. New shares will be issued, making your shares less valuable.

 

Watch out for bad people:

I’d say about half of the self proclaimed entrepreneurs I’ve met were good people and the other half were narcissistic sociopaths.

If you get in bed with a scum bag….

For a software developer / co-founder CTO to get anything positive out of startup equity everything has to go perfect. So much of that is completely out of your hands. Plus there is the chance you are being lied to by a sociopath Founder-CEO. So even if things do go perfectly for the company you may still get crumbs and an unfavorable legal position. Scam artists tend to blow all their money quickly, funneling it into cars, drugs, and the next scam. That makes recovery problematic assuming you can even find them.

They drink their own kool aid….

Some startup founders get so obsessed with their idea it becomes dangerous. At first nobody else believes in them or their idea which is psychologically taxing. So they invent a persona to carry on.  But it can get out of control. In a few cases I’ve seen startup founders justify breaking their word, screwing over their customers, and treating people like trash all for the sake of the company’s mission.

To me, just because you have a startup, you are in debt, and you haven’t had a paycheck in six months it doesn’t mean you are allowed to lie to people, be unethical, or act like you are above the law.

Tips for checking out potential co-founders / startup employers:

  • Weed out the dreamers by asking what their budget is, what their time commitment is, what their plans are for dealing with competition.
  • Watch how they treat others, someday they will treat you the same.
  • In the United States – check out their UCC filings. These are state level publicly registered debts. These kinds of debts are important in bankruptcy.
  • See if they are operating any other businesses.
  • Check their residence and see if the purchase date, location, etc lines up with their story.
  • Check their social media presence, do a google search, etc look for any red flags.
  • Ask who their attorney is. If they don’t have one, they have failed the test.
  • Other names will likely come up in your searches, perform similar lookups on their “known associates”.

Understand what Control and Profit are:

If your partner buys a Tesla with company funds, do you get to drive it?

Control directly translates to personal enrichment even if the company is struggling and taking on debt.

Let’s say you own 20% and your partner owns 80%. Your partner (who has control) can lease a Tesla on company money and keep the car all to themselves. Even though you have 1/5th of the stock you don’t get to drive it. You don’t even get the spare tire! All you can do is look at it with contempt.  In fact, you can be fired by your “partner” at any time since they have control.

Profits are not for minority owners:

Perhaps your agreement clearly says if the company makes a “profit” this year you are entitled to 25% of it! When it comes to zeroing out profit for tax purposes consider how useful control is. Your partner may decide it is essential to now have a yacht for entertaining prospective customers, to hire their family members, and to give themselves a raise, etc, etc. There goes all the profit. This may mean the end of your relationship with them, but what do they care, they now have the money to easily replace you.

Navigating the “shark tank” of equity for software development:

Kinds of crappy offers out there:

  • 1.3% vesting over 4 years, starting salary $900/month.
    • This is below minimum wage which is illegal.
    • Anything below 5% should command a market salary, in exchange for the career risk the software developer is taking on by accepting the assignment!
  • 20% equity for developer, 80% equity for CEO/founder, no wages
    • Developer is allowed to freelance 12 hours/week on the side to pay their bills. Wow that’s really generous.
    • Due to possibility of dilution and lack of control this arrangement leaves nothing but crumbs on the table for the developer.
    • What if the CEO/founder gets bored or loses focus or hires their stupid nephew who “knows computers”? The software developer cannot recover their investment!
  • Zero pay and no paperwork, just come and “be a part of the madness”. WTF!!!
  • Developer was asked to join as a “co-founder” and contribute $24k to a company which had a launch pending in 4 weeks. Turns out the splash page had been there for months, and after 6 weeks nothing changed. This was a scam.

You have two choices when evaluating an offer:

  1. Get your own lawyer to review the documents. It is worth every penny (provided you set limits and expectations up front). I’ve found when negotiating, saying “my lawyer is asking for X” is much more credible and likely to be accepted than simply saying “I would like X”.
  2. Sign the paperwork blindly and risk that years later you’ll suddenly get screwed out of something you love and poured countless hours into.

Avoid complex deals:

I used to think creativity was useful to apply everywhere, including legal agreements.

What I have learned is when it comes to business arrangements, keep it simple and stay with established patterns. If the agreement needs to be out of the ordinary then there is probably something else majorly wrong with the situation.

Use an LLC or incorporate:

I’ve casually used the term “partner” in this post, but you never want to be in a “legal partnership” which means joint liability in the eyes of the law. In a “legal partnership”, if your partner wrecks the Tesla, say smashing it into a liquor store, you are liable for the damages too.

Always do business through a LLC (limited liability company) or other corporate entity that provides personal protection.  LLCs are easy to create online at the Secretary of State’s office for your state without a lawyer.

Then when a project goes sour, you are not a “partner” who is jointly liable for the debts.

Use each LLC once and never again (like kleenex, use and throw away).  This is because the problems of any past venture will continue to follow the LLC.

This basic step will easily and cheaply insulate you from most of the problems of a failed venture.  Doing business through a LLC means everything you sign should be as “joe developer LLC” and all paychecks, etc. should read the same.

 

More Reading:

 

I hope you enjoyed this post. If you have a horror story related to startup equity that you’d like to share please contact me or leave a comment below.

Posted in Business | Tagged , | Leave a comment

Django Tricks for Processing and Storing JSON

In this post I’ll show a few tricks I use to make JSON fit into Django more seamlessly. The first is a lesson on coding that everybody should know.

Parsing external JSON:

Whenever you take in JSON from “strange computers” (which is basically any computer) it works most of the time.

As C3p0 said: R2D2, you know better than to trust a strange computer.

The following example code does two things that are super important when it comes to processing JSON in python.

  1. Handles the exception in case the JSON is not valid. This happens all the time with unescaped characters, server errors, etc. In the code below raw_data is a string that allegedly contains valid JSON. Not if, but WHEN raw_data is some random invisible UTF-8 character, you want to recover from it, and you want to log what you know about the problem.
  2. The parser keeps the keys in order. Without the object_pairs_hook=OrderedDict argument, json_data’s internal dictionary keys will be in whatever order your Python interpreter felt like that day. I’ve found for some types of JSON data order does matter and most systems that claim to emit ‘ordered JSON keys’ don’t realize that isn’t how JSON works.
try:
	json_data = json.loads(raw_data, object_pairs_hook=OrderedDict)
except JSONDecodeError:
	logger.exception('Error when parsing JSON, raw data was ' + str(raw_data))
	raise ExternalAPIException('Unable to do my work! Invalid JSON data returned.')

Now that you’ve seen a basic example of processing JSON, the rest of this post will be about storing JSON and integrating it nicely into the Django admin.

Some background on Django + JSON:

If you are using Postgres life is easy with JSON and Django because way back in Django 1.9 (December 2015) the Postgres only models.JSONField() came out.  Prior to that you’d be using a TEXT field to store JSON.

MySQL introduced a JSON type in version 5.7 but Django doesn’t support it out of the box as of 2.1.  Django requires you to implement JSON storage in a TextField (MySQL type longtext) and leave it at that.  I’m ‘guilty’ of using MySQL TEXT to store JSON on a few older projects and it works fine for me.

With the MySQL native JSON field you get additional JSON validation at the database level (which you should be doing at the API level already), optimized storage (yes please!), and ability to use non-standard SQL to index and query values inside the JSON. The last part is interesting but also makes me cringe a little because it is NoSQL wrapped inside an SQL database… Too much rope to get tangled up in.

For MySQL users there is another way…. if you really want to use the native JSON type in MySQL the Django-Mysql package is available.

Consider the following contrived model (models.py):

# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from django.db import models
from django_mysql.models import JSONField


class BookExample(models.Model):
    id = models.BigAutoField(primary_key=True, editable=False)
    name = models.CharField(max_length=100)
    detail_text = models.TextField()
    detail_json = JSONField()  # requires Django-Mysql package

    class Meta:
        managed = True
        db_table = 'book_example'
        verbose_name = 'Book Example'
        verbose_name_plural = 'Book Examples'

For illustrative purposes it has a TEXT based column and a JSON based column.

 

Running makemigrations + migrate will generate the following MySQL table:

CREATE TABLE `book_example` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
  `name` varchar(100) NOT NULL,
  `detail_text` longtext NOT NULL,
  `detail_json` json NOT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB;

 

Django admin wiring (admin.py):

# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from django.contrib import admin

from example.models import BookExample


class BookExampleAdmin(admin.ModelAdmin):
    list_display = ('name',)


admin.site.register(BookExample, BookExampleAdmin)

 

Now let’s fill in some sufficiently complex (contrived) JSON:

{
  "title": "Anathem",
  "authors": ["Neal Stephenson"],
  "publication_year": 2008,
  "description_sort": "Anathem is a science fiction novel by Neal Stephenson, published in 2008. Major themes include the many-worlds interpretation of quantum mechanics...",
  "chapters": [{
      "title": "Extramuros",
      "number": 1,
      "summary": "The story takes place on Arbre, a planet similar to Earth...",
      "page_count": 23
    },
    {
      "title": "Cloister",
      "number": 2,
      "summary": "Erasmas describes several buildings of the Concent, namely the Scriptiorium...",
      "page_count": 14
    },
    {
      "title": "Aut",
      "number": 3,
      "summary": "Erasmas describes the Mynster, which is a building housing the Concent's clock...",
      "page_count": 34
    }
  ],
  "language": "English",
  "page_count": 937
}

 

Here is how it would look in the admin using default settings:
 Django admin with JSON default

Both fields render as <textarea> fields (which are fully editable) and it is really hard to read the contents.  The first field is a plain textarea and it will accept any data. The second field has JSON validation wired to it so the form won’t go through unless the JSON is valid.

Let’s make the JSON look better in the admin and make it read only:

When I’m storing JSON in a database it is typically either:

  • 3rd party data associated with the record that needs to be kept but is rarely used.
  • A document format used by a front end tool or mobile app (designer tool, dashboard layout, or mobile app data sync).

It normally wouldn’t make sense to edit the raw JSON data in the admin. But it would make sense to edit other properties on the row and at the same time see a nicely formatted version of the JSON next to the other fields.

Here is how to make the JSON read only in the Django admin and format nicely.

First in the model, wire up a function to output the formatted JSON. This relies on the python Pygments package.

# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from django.db import models
from django.utils.safestring import mark_safe
from django_mysql.models import JSONField

# new imports!
import json
from pygments import highlight
from pygments.formatters.html import HtmlFormatter
from pygments.lexers.data import JsonLexer


class BookExample(models.Model):
    id = models.BigAutoField(primary_key=True, editable=False)
    name = models.CharField(max_length=100)
    detail_text = models.TextField()
    detail_json = JSONField()  # requires Django-Mysql package

    def detail_json_formatted(self):

        # dump the json with indentation set

        # example for detail_text TextField
        # json_obj = json.loads(self.detail_text)
        # data = json.dumps(json_obj, indent=2)

        # with JSON field, no need to do .loads
        data = json.dumps(self.detail_json, indent=2)

        # format it with pygments and highlight it
        formatter = HtmlFormatter(style='colorful')
        response = highlight(data, JsonLexer(), formatter)

         # include the style sheet
        style = "<style>" + formatter.get_style_defs() + "</style><br/>"

        return mark_safe(style + response)

    detail_json_formatted.short_description = 'Details Formatted'

    class Meta:
        managed = True
        db_table = 'book_example'
        verbose_name = 'Book Example'
        verbose_name_plural = 'Book Examples'

And in the admin file:

# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from django.contrib import admin

from example.models import BookExample


class BookExampleAdmin(admin.ModelAdmin):
    fieldsets = [
        (None, {'fields': ['name']}),
        ('JSON', {'fields': ['detail_json_formatted']}),
    ]
    list_display = ('name',)
    readonly_fields = ('detail_json', 'detail_json_formatted')


admin.site.register(BookExample, BookExampleAdmin)

Now the admin has a nice easy to read formatted representation of the JSON complete with inline styles:

Django admin json pretty print

 

Other ideas for using JSON with Django:

Posted in Application Development | Tagged , , | Comments Off on Django Tricks for Processing and Storing JSON

My Answer To: How To Stay Current in Software?

I had a developer write to me a few weeks ago asking how to stay current. With their permission I’ve published their question and my answer with personal details omitted.

Question from a new developer:

How do I stay current? I am a new developer at a small software company. In my current position, I am a support developer, mainly database scripting. I worry that my skills from college involving object oriented coding are falling behind. What is a good way to keep practiced? I have a drive to be very good at all types of projects, yet I know very little of the full stack experience.

My Reply:

Side projects are a great way to “practice” coding. Pick an interest of yours and go for it. Don’t let your employer be the only source of skills you gain. After awhile your work tasks should get easier. You’ll end up with free mental cycles to read articles and view videos during your work day (or on your lunch break) that fit into your side projects.

If you work for a place that discourages side projects or personal learning it is time to get another job.  Do yourself a favor by being respectful and tactful when making your move.

Their reply:

Thank you. I figured as much. I have picked up side projects that relate to my work, like automation tools for uploading report to a reporting server. As well as an XML generator specifically for our programs report import process. Sounds like I am doing what I can. I appreciate the feedback.

My follow up:

Hmmm…. I was thinking of something less work related and more for the joy of it – like a new language or framework.

They say you are what you eat.  In software it is the same thing – you are what you do.

You won’t get hired for 3D graphics if you’ve never done it before. If you are really good with Crystal Reports and that is what you mess with in your free time, chances are that is all you can get hired for. If you yearn to build 3D games, mess with 3D games in your free time and see where it takes you.

I was always drawn Python but wasn’t allowed to use it at work for years. I tried to slip it in here and there but when you are working on a mature code base it isn’t possible.  Slowly but surely I tinkered with it in my free time. Then I taught myself Django which meshed really well with my other web experience and the professional work started coming.

Fast forward to current times and I use Python on about half of my professional projects!  I strongly considered Ruby/Rails too. I spent a lot of time experimenting with RoR but it never went anywhere in terms of gigs. It wasn’t time wasted because my overall mental context of coding and web frameworks was expanded.

In terms of the future in 2018… “Go” is an upcoming language that looks interesting. There’s always mobile apps and Node/JavaScript/React/Angular to look at.

Make your side projects feel fun, you know, like you want to code all weekend 🙂

Posted in For New Developers, Work | Tagged , | Comments Off on My Answer To: How To Stay Current in Software?

Windows 10 Updates Failing with MEMORY_MANAGEMENT due to DiskCryptor

Having problems updating to Windows 10 1703, 1709 or 1803? If you have DiskCryptor installed the updates will fail with the unhelpful message `MEMORY_MANAGEMENT`. Then you’ll be trapped in a kind of Windows Update purgatory:

  • Windows downloads update (30 minutes – 1 hour)
  • Windows installs the update (30 minutes)
  • Windows reboots, chugs for a long time, then the update fails (30 minutes)
  • Windows reboots and rolls back the update (30 minutes)
  • Windows reboots… then goes back to step 1 and starts to download the update again!!! Ahhh!!!

On my Windows box which I use for internal QA this was taking about 2.5 hours round trip. It only affords about 30 minutes of usable time on the desktop. I tried various fixes and hacks to get it to go through. Nothing worked until I stumbled across this and this that mentioned DiskCryptor was the problem.

As it turns out DiskCryptor hasn’t been updated in almost 4 years so it is probably time to find an alternative.

Here is how I was able to get around the MEMORY_MANAGEMENT error blue screen while installing Windows 10 update 1703, 1709 and 1803:

  1. FIRST – TAKE A BACKUP OF THE CONTENTS OF YOUR ENCRYPTED DRIVE!!! I ran into problems getting DiskCryptor to re-install which was more than a little scary but I did get it to work… If you follow these instructions, do so at your own risk. Make sure you have a backup before starting. These steps worked for me given the unique set of programs & hardware I have but may not work the same for you. You could get locked out of your encrypted disk if it fails to reinstall correctly. Note – I’m not encrypting my boot partition, nor my entire drive, I only encrypt a small portion of my drive which I mount when I need it.
  2. Note the current version of DiskCryptor something like 1.1.846.118.
  3. Download that version of DiskCryptor here so you are ready to reinstall it later.
  4. Uninstall DiskCryptor (Control Panel -> Programs and Features -> Uninstall a Program…) – this is the point of no return – READ STEP 1!
  5. Stop the Windows Update service (Control Panel -> Administrative Tools -> Services), scroll down to the Windows Update, right click on it, then click Stop. This can take several minutes. You may need to reboot first and try this, or set it to disabled, then reboot, but it needs to be off for the next step. If you do disable it make sure to re-enabled it after everything is done.
  6. With Windows Update service stopped, delete everything in C:\Windows\SoftwareDistribution -> that is where windows downloads all the windows update files before it installs them. These downloads can get corrupted. I had 61K files in that folder! Took it several minutes to complete. So at this point, maybe take a breather, stretch, eat a cookie – do something to help relieve all the stress Windows is putting you through at the moment.
  7. Reboot and run the Windows Update Troubleshooter (which you can download here).
  8. Then run the Windows Update Assistant (which is also a small download found here) and let it do the update.
  9. After a few reboots and tense moments – the update finally went through for me!!!
  10. Next re-install Disk Cyptor, but right click on the installer and select Run as Administrator. It may come up with an error like “error occurred when installing driver”…
  11. Reboot.
  12. Now run DiskCryptor like you would normally, but it may ask you to reboot (WTF????- PANIC!!!)…. select reboot.
  13. When the desktop comes back up – uninstall DiskCryptor again. Do not start it again, as it will just ask you to reboot.
  14. Now install it as Admin again, reboot… cross fingers.
  15. When the desktop comes back up, run DiskCryptor, and for me it worked….!

Steps 12 – 15 seem kind of crazy, but I went through this process on two machines with DiskCryptor and it worked both times.

Given this fiasco I’ll be looking for an alternative to DiskCryptor – here is a list of disk encryption software, many look promising. On Mac OS and Ubuntu encryption is built in, all you need to do is enable it. Unfortunately on Windows you are on your own.

In the mean time if you are interested in protecting your online identity, check out this post on some of the basics.

Posted in Sys Admin | Tagged , | 1 Comment

Django – Correctly Wiring to AWS CloudFront for Static and Media Files

When it comes to “How To Setup a CDN for Django” most people suggest the following “half way” setup that can lead to stale cache problems and slowness in the admin.

  1. Configure Django to upload media and static files to S3. That is done by installing django-storages and boto, then setting DEFAULT_FILE_STORAGE to your S3 bucket.
  2. Configuring your AWS credentials.
  3. Setup your CloudFront distribution to pull from your S3 bucket.
  4. Set Django’s MEDIA_ROOT and STATIC_ROOT settings to point to the public URL of the CloudFront distribution.

For simple sites this can be an okay way to go. Wiring it up takes just a few minutes and you get automatic backups of all uploaded media. This approach is required in some hosting environments, such as Heroku, where you don’t get a permanent local disk to store files on.

I’m not sold on it though…

Using S3 as the Media / Static Root Can Be a Bad Idea:

1) CDNs cache everything aggressively AND pass expires headers to clients that tell them to cache aggressively too.

Unless you are actively managing the state of the CDN cache (which takes forethought with CloudFront) if the content of a file changes some of your users are guaranteed to get the stale version and see a messed up version of your site.

This will happen pretty much every time you push new code.

There is a mechanism in CloudFront to invalidate files, but if you do over 1000 operations per month it starts costing you, and isn’t instantaneous.

For the kind of high profile Django sites I work on, it would be detrimental if the user got to a page that was a franken-build mix of new and old assets because of stale cache.


2) Write performance to S3 is slow.

Many of the sites I’ve worked on use VersatileImageField to generate a handful of cropped / resized versions of images after they get uploaded through the admin (large, medium, thumbnail). When S3 is the default file storage the admin runs slow because of the intensive IO work needed to generate the images and push them to S3. With VersatileImage field, every time you save the model, it revalidates all those files, which causes it to scan the S3 bucket (really slow).

So if you are doing a lot of interesting things with your media files, your site will bog down while Django talks back and forth to S3.

 

How I Setup CloudFront with Django:

1) Don’t use S3 to store files…

Instead configure MEDIA_ROOT and STATIC_ROOT to be on your server like you would normally.

There’s a twist for STATIC_ROOT, see below.

2) Setup your server as normal, like you would without CloudFront, so it serves /media/ and /static/ on its own, but do set some extra headers.

Configure your server so /media/ and /static/ are being served as normal from your domain. It is helpful to set the CORS header Header set Access-Control-Allow-Origin "*" so things like custom fonts work correctly.

Example Apache 2.4 configuration for serving media and static files, with CORS header:

# STATIC DIRECTORY
Alias /static /var/mysite/static
<Directory "/var/mysite/static/">
   Options +FollowSymLinks -Indexes -MultiViews
   Require all granted
   # add the CORS header so the AWS CloudFront distribution can serve all files, including fonts
   Header set Access-Control-Allow-Origin "*"
</Directory>

# MEDIA DIRECTORY
Alias /media /var/mysite/media
<Directory "/var/mysite/media/">
   Options +FollowSymLinks -Indexes -MultiViews
   Require all granted
   # add the CORS header so the AWS CloudFront distribution can serve all files, including fonts
   Header set Access-Control-Allow-Origin "*"
</Directory>

Note: this is only an option if your hosting configuration has access to a local partition, not always the case (Heroku for example).

3) Setup two CloudFront distributions using your server as the origin.

Create one distribution for the media files and a second distribution for the static files. This improves download performance during page load.

4) Add a variable into the static path / url that busts the cache for each deploy.

AWS calls this strategy Using Versioned Object Names.

The trick is to configure Django’s STATIC_ROOT and STATIC_URL so they include a configuration variable that gets updated every deploy.

STATIC_ROOT = '/var/mysite/static/' + BUILD_VERSION + '/'
STATIC_URL = 'https://{static-distribution-id}.cloudfront.net/' + BUILD_VERSION + '/'

When the deploy happens and collectstatic is executed all the files go into a new folder just for that version of the app. Those files are totally new to the CDN and all your users. When your users hit your site, they are guaranteed to get the new version of all static files.

Note, for media files I’m not setting up a dynamic path. This is okay for most uses cases. If a user uploads a file or image with the same name Django will take care of giving it a unique name.

Example settings.py file:

BUILD_VERSION = 'mysite-1.0'

#############################################
# MEDIA & STATIC FILES
#############################################

# store media files locally on the server
MEDIA_ROOT = '/var/mysite/media/'

# Serve media files from the CloudFront distribution
MEDIA_URL = 'https://{media-distribution-id}.cloudfront.net/'

# When collect static runs, it will copy the files to a path in appending the BUILD_VERSION
# this way each deploy gets a fresh URL folder, no need to worry about invalidating stale caches in the CDN
STATIC_ROOT = '/var/mysite/static/' + BUILD_VERSION + '/'

# serve static files from the CloudFront distribution
STATIC_URL = 'https://{static-distribution-id}.cloudfront.net/' + BUILD_VERSION + '/'

This requires updating BUILD_VERSION before each deploy. I like having that as a pre-deploy step.  I update that value at the same time I minify the CSS/JavaScript files, then commit it all. Makes a nice marker in the commit history along with a new version tag.

For systems with automated builds updating that variable can easily be part of the overall build / test / deploy process.

How It Works:

Initially, your site’s CSS file would live at:
https://mysite.com/static/mysite-1.0/css/mysite.min.css

But it would be served through the CDN at:
https://{static-distribution-id}.cloudfront.net/mysite-1.0/css/mysite.min.css

As part of the next deploy, you’d change BUILD_VERSION to mysite-1.1. After the deploy the next page request to your site would generate a request for the CSS file at:
https://{static-distribution-id}.cloudfront.net/mysite-1.1/css/mysite.min.css

The CDN would pull it off your server as a totally fresh file and cache it across the entire network.

Why I like it:

I opt for a simple solution that works well all of the time.

Other solutions to busting the cache do things like adding a query string parameters to CSS / JS files. For example:
https://mysite.com/static/css/mysite.min.css?version=mysite-1.0.1

In practice, that works okay but not 100% of the time because some proxy servers ignore the query string for the purposes of the cache! This is true even if you are forwarding query strings with CloudFront. That is the kind of bug you only run into once in a blue moon and it is impossible to replicate.

One minor drawback is, each time the site is deployed collect static copies all the files to a new folder. At some point someone or something will need to cleanup the old folders, but only when you know it is safe to do so. Storage is cheap and the static directory is usually pretty small so I don’t see this as a huge problem.

Where not to use this idea:

If you have load balancers with multiple servers this solution may not be for you.

If your media files are only changed through the admin then it may work, but you still have to make sure the bucket’s origin points to a location on your side that will always see the newly uploaded files. That can defeat the purpose of load balancing…

If you have lots of users uploading media content across all your servers it would be better to send the new uploads directly into S3 (or save locally at first then push to S3 in the background). In that case it would be important to avoid file name collisions – where user A uploads cat.gif and user B uploads another cat.gif in the same instant, and user A’s cat.gif gets overwritten.  In theory Django will automatically check for file name collisions when uploading new media, but S3 has such high latency I’d take extra precautions.

Another approach would be to setup an NFS share that all your load balanced servers tie into for media storage. Then you could continue with the setup above. But setting up an NFS adds complexity and another potential point of failure.

Thank you Alberto for bringing up this scenario.

Closing thoughts…

No matter what it is absolutely essential to setup a CDN. It is dirt cheap, it will make the site load very fast for all your users no matter where they are, and doing so is based on best practices. Just don’t cause bugs you will never see or be able to replicate due to a stale cache…

Some tools I use to test website performance:

 

Posted in Application Development | Tagged , , | Comments Off on Django – Correctly Wiring to AWS CloudFront for Static and Media Files

Laravel Backpack Crud Explained and Reviewed

Recently had a project where I needed to get a CRUD admin up and running quick. My go to framework for CRUD is Django Admin. However Python/Django isn’t always an option for various technical or political reasons. In this case it had to be PHP. So I looked high and low for an out of the box framework that would allow editing of tables and thankfully Backpack Crud (docs, github) was there.

What is CRUD?

  • C – create
  • R – read
  • U – update
  • D – delete

I didn’t really want to roll my own admin interface (been there done that), so I went ahead and installed Backpack Crud.

After you get Composer installed, Laravel installed and Backpack Crud installed, you should be ready to start adding models / controllers.

Backpack Crud has pretty solid documentation included a nice getting started page. I’m doing this write up to to document the extra things I ran into (see example below).

First off, as of January 2018, I ran into dependency conflicts with Laravel 5.5 and Backpack Crud 3.3. I ended up going with this in my composer.json file:

"backpack/base": "0.7.24",
"backpack/crud": "3.2.27",
"laravel/framework": "5.4.*",

How to setup a Backpack Crud model:

  1. Create a new model/request/controller php artisan backpack:crud {name} (name is singular).
  2. Setup model, at a minimum the $fillable property.
  3. Setup request class with validation rules and messages.
  4. Setup controller with fields, columns and filters.
  5. Add a new route in web.php referencing the controller.
  6. Add to link to sidebar (sidebar.blade.php) to new controller.

1) Run the generator:

Say you need CRUD for table of colors with fields: name, sort order and hex code.

First run:

php artisan backpack:crud Color

That will auto generate the model, request and controller files. Then it is up to you to customize the contents of each file. You will also need to setup migrations for the new models.

2) Setup Model (app/models/Color.php):

The model is a straightforward Eloquent model with use CrudTrait;.

What tripped me up at first was you have to set the $fillable property on the model or else nothing shows up! So at a minimum you need to list all the columns you want the user to edit in $fillable.

<?php
namespace App\Models;

use Illuminate\Database\Eloquent\Model;
use Backpack\CRUD\CrudTrait;

class Color extends Model
{
  use CrudTrait;

  protected $table = 'colors';
  protected $fillable = ['color_code',
    'name',
    'sort_order',
    'hex_code'];

}

3) Setup Request (app/Http/Requests/ColorRequest.php)

The validation rules for each file go under rules() and the friendly names for the columns go under attributes().

<?php
namespace App\Http\Requests; 
use App\Http\Requests\Request; 

class ColorRequest extends \Backpack\CRUD\app\Http\Requests\CrudRequest {

    public function authorize() { 
        // only allow updates if the user is logged in return \Auth::check(); 
    } 

    public function rules() 
    { 
        return [ 'name' => 'max:50',
            'sort_order' => 'required|numeric|min:0|max:9999999',
            'hex_code' => 'max:6',
        ];
    }

    public function attributes()
    {
        return [
            'name' => 'Name',
            'sort_order' => 'Sort Order',
            'hex_code' => 'Hex Code',
        ];
    }

    public function messages()
    {
        return [
            //
        ];
    }
}

4) Setup Controller (app/Http/Controllers/Admin/ColorCrudController.php)

If you are setting up lots of controllers and copy / pasting watch out for the path to StoreRequest and UpdateRequest, it needs to match. IntelliJ hides the namespace and use section by default so I didn’t see that. I wasted some time tracking down some really strange errors with form validation labels.

The setFromDb() command is what wires up everything, but I found I had to comment that out and manually add all the fields (add/edit page) and columns (listing page) since the schema I was handed had the columns in a different order than made sense on the admin screens. For a really simple table setFromDb() is probably fine though.

<?php

namespace App\Http\Controllers\Admin;

use Backpack\CRUD\app\Http\Controllers\CrudController;

// NOTE: change the requests to match your own file names if you need form validation
use App\Http\Requests\ColorRequest as StoreRequest;
use App\Http\Requests\ColorRequest as UpdateRequest;

class ColorCrudController extends CrudController
{
    public function setup()
    {
        $this->crud->setModel('App\Models\Color');
        $this->crud->setRoute(config('backpack.base.route_prefix') . '/color');
        $this->crud->setEntityNameStrings('Color', 'Colors');
        $this->crud->setFromDb();
.....

5) Setup routes (web.php)

Route::group(['prefix' => 'admin', 'middleware' => ['admin'], 'namespace' => 'Admin'], function()
{
    CRUD::resource('color', 'ColorCrudController');
    // … add more CRUD:resource lines for your models here
});

6) Add link to side bar (resources/views/vendor/backpack/base/inc/sidebar.blade.php)

<li><a href="{{ url(config('backpack.base.route_prefix', 'admin').'/color') }}"><i class="fa fa-table"></i> Colors</a></li>

 

What is great about Backpack Crud:

  • It uses Laravel which is a well thought out PHP web framework that takes a lot of the pain out of PHP development. Laravel’s ORM Eloquent (also great) is central to Backpack Crud. Laravel also has excellent documentation and tutorials, including their Laracasts so it is really easy to get started.
  • Backpack Crud understands Foreign Keys and can wire them to select2’s on the UI (even driven by ajax).
  • You get a lot of control over how thing are displayed and formatted. It allows custom fields and columns.
  • Has a mechanism for filtering data in the listing.
  • Has a built in way to export the listing data to various formats (CSV, PDF, etc).
  • Has a way to make a table read only.
  • Has a user / permissions model (I did not play that part of it too much).
  • A nice selection of baked in UI widgets (date picker, WYSIWYG editor, select2, etc).
  • Has a built in revisions tracker for viewing a history of each record.
  • Good documentation.
  • Backpack Crud does come with a paid licensing model (free for non-commercial use only). It’s affordable and I like the fact that somebody is getting paid to maintain it.

Where I’d like to see Backpack Crud improve:

  • When you wire up a model, you have to add routes and a menu link yourself. That was confusing at first because nothing was showing up. Would be nice if the menu would automatically include the models that have `use CrudTrait;`.
  • Even though Backpack Crud understands foreign keys, you can’t edit child and parent records on the same screen. Django admin has a feature called inlines that lets you do that and it’s been available for over a decade. It is one of my favorite Django features. Yes inlines can get way out of hand if you have lots of child rows or more than a handful of relationships to manage. Still it is a major time saver and works really well.
  • Getting a model wired up involves a lot of verbose code, some of which is repetitive or redundant. It requires you to declare the same field in several places (model, controller, requests, routes, and menu link). Seems like all Backpack Crud needs is a configuration wrapper around the existing functionality to make the implementation faster. That’s how Django admin works – you just give it a list of fields, link it to a modal, and it does the rest.
  • A built in event log of who changed what, independent of revisions would be pretty neat. I ended up rolling my own, which wasn’t too bad because I was able to leverage Eloquent Observers.
  • After we started using it we found select2 drop downs powered by a model were not sorting correctly. There is no way to specify default the sorting on a select2 (1-n relationship) drop down. In Laravel you can add a Global Scope that will add a default sort to all queries for a given model. That works (sort of), but it did cause conflicts on listing pages with ajax enabled and pagination (at least for me on MSSQL).  So my solution was to convert all the select2 drop downs to select2_from_ajax and wire my own custom endpoints / routes / queries that had the right sorting. So at least there was a work around.
  • The listing pages use DataTables.js which is a powerful but somewhat arcane jQuery based table plugin.  For any kind of serious app you’ll probably end up having to override /backpack/crud/list.blade.php   (copy it from vendor into resources/views/vendor/backpack/crud/list.blade.php)  and hack your way through its setup.
  • Related to DataTables.js, one gem I found was the saveState option which automatically remembers the current page number, page size, and sort order for the table – this is essential for a real life application with thousands of rows. Otherwise after adding or editing a record users are always dumped to page 1 with the default sort options (and that is really frustrating for them).
  • Another DataTables.js feature I was asked to implement was a Jump To Page button where the user could type in a page number and it would take them straight to it. This tutorial was really helpful.

 

Posted in Application Development, Code, Data | Tagged , | Comments Off on Laravel Backpack Crud Explained and Reviewed

How to Override a Composer Dependency so it Pulls From Your Branch

Ran into a situation this week where I needed to fix and slightly customize a dependency of a dependency. This was for a Laravel project that uses Composer to manage dependencies.  Since I’m using Composer everything in /vendor is managed by Composer and is off limits to code changes.

Composer

I need to get things launched – with a fix bugs first attitude. Who has time to wait for a pull request to get approved? Not me.

Thankfully there is a way to tell Composer to override a dependency and use your fork instead of the main provider’s.

1) Make a fork of the repo. 

Do your amazing high touch changes in a branch in your fork. Going forward in this example, let’s say your branch is called myfix.

Side note – in my case, the original project was in github, but I wanted my fork to live in bitbucket. So I followed these steps. It’s not technically a fork but it is still connected to the original repo.  Just make sure the fork / copy has the exact same project name as the original.

2) Update your composer.json file.

"repositories": [
    {
        "type": "git",
        "url": "https://bitbucket.org/myuser/thedependency/"
    }
],
"require": {
    "originalowner/thedependency": "dev-myfix as 5.3.0"
},

The repositories section is where you tell composer about your repo (where the myfix branch lives). Composer will automatically scan for branches in that repository.

The require section “dev-myfix as 5.3.0” part was confusing to me at first so I’ll explain what it is doing.

Your branch is named myfix, but trust me, write in dev-myfix.  That tells composer it is a custom branch (dev- is a composer convention).  Then after dev-myfix the as 5.3.0 part creates an inline alias that allows Composer to satisfy the parent dependency, or anything else in your project that needs thedependency 5.3.0.   In this case 5.3.0 is an example. Make sure the version alias you type into your composer.json files matches what your project needs. You can find that out by running composer show.

3) Re-install dependencies:

  1. Run composer clearcache otherwise it will supply thedependency from a copy in your local cache that points to the original repo. That’s not what you want.
  2. In my case, deleting the composer.lock file was safe and this cleared the way for the next step.
  3. Run composer install

 

Posted in Application Development | Tagged , , | Comments Off on How to Override a Composer Dependency so it Pulls From Your Branch

An Awesome Monitor Riser For My Desk

I always wanted a monitor riser for my desk that looked good, was made of natural wood and had internal shelves.  After searching online and coming up disappointed I decided to build my own.

Here it is:

monitor riser for software developer desk

The extra shelves instantly made my life better. The over organizer in me loves having dedicated spots for calculators, note books, noise cancelling headphones, tape measure, flash light, coasters, paper weights, todo list, etc…

monitor riser hides hdmi cable

The design allows me to tuck the latop’s HDMI cable under the first shelf.  It also has room to stash the wired keyboard that goes to my tower out of sight.

Stats:

  • Width: 28″
  • Height: 7 3/4″
  • Depth: 9 3/4″

When building this I took several measurements to get the height just right so my monitor would be eye level with my desk in either sitting mode or standing mode.

Made of cherry wood.

Finished with Tried and True Original Wood Finish (Beeswax and Linseed oil).

My History With Monitor Risers is Ugly:

At past desks I’ve done all kinds of hacky things to get the monitor height just right. This included text books, reams of paper, or ugly looking hunks of plastic. Here was my workstation in 2006:

I recall just loving this desk with the 3 monitors. Today in retrospect it is a little sad looking…. This was at a startup so the fact it was cheap and did the job was smiled upon culturally.

Ergonomics Is Really Important For Software Developers:

The more time you spend at your desk the more it will strain your wrists, neck and lower back.  Having the right setup is important for your health. I’m not an ergonomics expert, but here is what works for me:

  • Herman Miller Embody chair
  • Sit stand desk which I elevate daily.
  • Monitor at eye level (in both sitting and standing mode), which this monitor riser solves or me.
  • Wrist pad for my keyboard.
  • Wrist pad for my mouse pad.
  • A daily walk / break away from the machine.
  • Regular light exercise /  yoga.

You may not need a monitor riser like mine if your monitor’s height is adjustable. Even if my monitor was adjustable, I’d still want the shelving which comes in super handy.

A Desk Should Be Ergonomic, Functional, and Beautiful:

I spend upwards of 50 hours per week at my desk.  Might as well make it as appealing as possible, right?

This new monitor riser has added utility with the shelves, but it is also beautiful. I’m a nerd and a fan of Star Trek.  I was going for the lines of the bridge of the Enterprise NCC-1701-D, and touch of the LCARS user interface. I think I captured both and this piece would work okay as a prop on TNG.

monitor riser for software corner detail

Aesthetics are often overlooked in office environments. I’m not a psychologist and I don’t have a peer reviewed study to quote but my hunch is a better looking office no doubt leads to job satisfaction and higher productivity. That’s how I justify my Millennium Falcon 5 piece canvas set that hangs on the wall:

software developer office Millennium Falcon

Inspiration:

I did a lot of looking around for a monitor riser with internal shelves and a cool design.

This one from CaseBuddy (Australia) is elegant but has no shelving:

case buddy monitor riser

This one from StudioHalf (The Netherlands) was my ultimate inspiration.

studio half riser

studio half riser 2

I went with 100% cherry (as opposed to fir / pine) but kept the exposed end grain design. My version has taller shelves and more aggressive angles and curves. I figure I spent about $80 in wood and it took me 8 hours total.

Hope this inspires you to make your desk ergonomic, functional, and beautiful!

Posted in Fun Nerdy, Work | Tagged , | 1 Comment

How to require SSL when connecting to MySQL on AWS RDS

With MySQL you can opt to connect to the database using an encrypted connection. This option is important to consider on AWS if you are using RDS for your MySQL database. The network between your EC2 server and the RDS server is controlled by Amazon. You don’t know who has access or what is going on in between. Enabling an SSL connection to MySQL does come with a small performance penalty. Back in 2013 I did some benchmarking on this.

First, get the CA file from AWS:

To enable an SSL connection to RDS for MySQL the first step is to download the certificate authority (CA) file from Amazon which can be found here. You may also want to read the AWS docs on the subject.

To make sure your MySQL connection is done over SSL you need to supply the CA file when connecting. It is also a very good idea to configure the MySQL user to require SSL connections. Examples of both are shown below:

When connecting via command line:

mysql -h mydb.rds.amazonaws.com
--ssl-ca=/home/myuser/rds-combined-ca-bundle.pem --ssl-mode=VERIFY_CA
You may have seen posts mentioning --ssl-verify-server-cert, but a new option --ssl-mode has been added in MySQL 5.7 which is more flexible. I'm using VERIFY_CA because it not only requires SSL, but also makes sure the CA matches.

In your .my.cnf, require SSL by setting the following:

[client]
user = myuser
password = ******
host = mydb.rds.amazonaws.com
ssl-ca = /home/myuser/rds-combined-ca-bundle.pem
ssl-mode = 'VERIFY_CA'

After you connect, to double that check SSL is being used:

Once you are connected, run the mysql status command and look for the SSL line:

mysql> status
--------------
mysql  Ver 14.14 Distrib 5.7.19, for Linux (x86_64) using  EditLine wrapper

Connection id: 		12388
Current database:
Current user:  		myuser@myinstance
SSL:   			Cipher in use is AES256-SHA
....
--------------

In addition, FORCE your applications to connect via SSL as well:

As the AWS docs point out, it is really smart to add the REQUIRE SSL option to all your GRANT statements. That way a lazy client that isn’t setup for SSL or has a misconfigured cert will get rejected.

GRANT SELECT ON mydatabase.* TO 'myuser'@'%' IDENTIFIED BY '....' REQUIRE SSL;

For example, to configure Django 1.11 to use an encrypted connection with AWS RDS:

1) Put the rds-combined-ca-bundle.pem file in the same folder as your settings file.

2) Add the OPTIONS directive below to the DATABASES section:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'mydatabase',
        'HOST': 'mydb.rds.amazonaws.com',
        'USER': 'myuser',
        'PASSWORD': '....',
        'OPTIONS':  {
            'ssl': {'ca': os.path.join(os.path.dirname(os.path.dirname(__file__)), 'settings', 'rds-combined-ca-bundle.pem')}
        }
    }
}

How to decide if you should encrypt your connection to MySQL?

If you are running a database of cat memes, you probably don’t have much to gain from enabling this. If your app handles personally identifiable information (PII) or sensitive personal information (SPI) then encrypting the data in flight between the app server and db server makes more sense.

Posted in Application Development, Data, Sys Admin | Tagged , | Comments Off on How to require SSL when connecting to MySQL on AWS RDS

DBAs are Out of Style and Now There’s a Hole In Your Database

The concept of a DBA – database administrator, has practically gone out of style as a full time job. DBA work, if it is being done, is handled by someone or something else, perhaps in a more vanilla way that works well enough for most systems. However, if the DBA work is being ignored then technical debt is silently piling up.

database administrator

There are still plenty of DBAs (about 120k) in the basements of large companies and government entities. It is just that growth in this area has really dropped off in spite of the fact it is a critical function of any data driven application. This is due to technological advancement (getting easier to run databases), and lack of standards (not anyone’s problem if the data is wrong from time to time). Still I think the Bureau of Labor Statistics is overly optimistic in its estimate that DBA positions will grow 11% between 2014-2024Compare that to the 1.1M software developer positions with estimated growth of 17% between 2014-2024.

So what exactly was a DBA and what did they do?

In the old days (80’s, 90’s, 00’s) the database administrators (DBAs) were in charge of all things database related.

  1. They made sure the database server was running correctly.
  2. They controlled how the data itself was structured so it could be stored efficiently.
  3. They made sure the maintenance scripts, upgrades and backups were running correctly.

Those three tasks are still pretty darned important to a successful system! Given that DBAs are not part of most software teams anymore, are we doing it right?

1) So who is now making sure the server is running correctly?

It used to be a lot harder to make a databases run smoothly. RAID arrays had to be custom configured. There were many arcane commands just to get the database to run on the network. Default block sizes had to align, custom configuration for memory, IO, etc.

Database setup has for the most part been handed to IT / DevOps / The Cloud. AWS RDS, for one example, makes databases an on demand commodity service (Amazon Aurora, PostgreSQL, MySQL, MariaDB, Oracle, and Microsoft SQL Server). In this regard, life couldn’t be easier for application developers to spin up a production ready database.

The default configuration is still something that needs to be checked over and tuned based on the amount of memory on the box but that can be done pretty easily in an afternoon by an application developer or other IT staff member.

2) So who is now making sure the data is ‘structured efficiently’?

In today’s world the words ‘structured efficiently’ amount to a loaded term. It could mean anything to anyone. But if you look around the room and don’t see anybody doing this, then it’s probably a good idea to log a task under technical debt and check into it.

It used to be that data storage was fairly expensive. Application developers were aware of the issue but not necessarily responsible for optimizing the cost of the system. DBAs had control here, and they earned their keep on that front alone.

With cheap storage, lots of RAM, and virtually unlimited bandwidth, efficiency at the level of the 1s and 0s is not as critical as it used to be for a general run of the mill application. To many business level decision makers, what makes something efficient is how soon it can launch. In today’s world the cost of data storage is essentially a rounding error in a company’s overall budget. So again, no real pressing need for a DBA. However, I think that without someone who knows their stuff at the wheel to make sure the data is sane, problems can crop up especially with maintainability.

There are a number of best practices to follow when it comes to storing data. It really varies by platform, for example with a relational database, indexes, normalization, and eliminating stale data is super important. In the NoSQL world though duplication of data is expected and plays into performance goals.

One thing I see all the time in naive database designs is treating what is really historical data as a source of authority. It leads to huge problems. For example, let’s say you have an invoice from last week that links to a customer. What if the customer changes their address tomorrow. Should the invoice change? No! The invoice is a historical record and still needs to reference the customer, plus the address they had at the time they placed the order. Noticing slowly changing dimensions is another way DBAs earned their keep. Unfortunately slowly changing dimensions isn’t taught in school or isn’t appreciated from the application development perspective.

Another thing that sometimes worries me is handing over the responsibility for the database schema to a web framework. Without a DBA application developers are expected to handle database design and schema changes. Frameworks like Rails and Django come with built in tools to do it for you. That approach works up to a point. But in fact, many frameworks will do really stupid things when it comes to JOINs or the ability to build reports that a true DBA would laugh at. In my opinion it is best to fully understand what is going on under the hood when a framework takes over the responsibility of your database structure.

Sometimes the DBA is helpful, but sometimes they are just doing stuff to make their job seem important. I recall one DBA required that all tables be prefixed with ‘t_’. All fields had to be prefixed with ‘f_’ and end with an underscore and a character that denoted the field’s data type (i for int, c for character, d for date time, etc). The DBA also wanted views prefixed with ‘v_’ and stored procedures prefixed with ‘sp_’.

For example, let’s say there was a user table with columns user id and username, the DBA wanted to see:

[t_user].[f_user_id_i]
[t_user].[f_username_c]

Compared this to how I’d typically do it, no prefixes or suffixes:

[user].[user_id]
[user].[username]

The column [f_user_id_i] is the kind of fussy standards DBAs were getting paid to enforce corporate wide in the late 90’s early 2000’s. I can appreciate standards, but only when they add value. Adding readily available meta data to the name of a thing is redundant. It also makes application code and SQL painful to read. So in this respect, I’m really happy I now get to use the shorter, more concise form [user].[user_id] in my code without a DBA lording over it.

3) Who makes sure maintenance scripts, upgrades and backups are running correctly?

Sadly, this one is often neglected unless there is a dedicated sysadmin / IT / devops team watching over it. Again if you look around the room and nobody is doing it, then you probably need to get it on the schedule fast. When it comes to an out of date database engine – what you have is technical debt of the most easily combustible kind.

It is becoming more and more popular for application developers to be responsible for running the systems they write. This is the approach I advocate for with my customers because I want to be responsible for what I create. I want to fix bugs first. I want to keep the system running perfect. Tasking application developers with db maintenance also allows the business to eliminate the DBA position. Sad, but again systems are getting easier to maintain. AWS RDS does maintenance patches for you in your sleep (for the most part).

Unfortunately a lot of business people look at a software system like a fridge – something you plug in and leave for 20 years. The reality it software systems are more like custom race cars, one of a kind, built for a specific purpose, and high maintenance. The data in your application is sort of like the oil in the car, it flows through the system, needs to be cared for, and should not be ignored or left leaking!

The DBA may be gone, or at least morphed into a part time system admin part time DBA, but someone needs to tasked with treating the data like gold.

Posted in Application Development, Data, Sys Admin | Tagged , , , | Comments Off on DBAs are Out of Style and Now There’s a Hole In Your Database