18 Dec 2015
I am spending some time learning Haskell. This is partly because I want to learn about functional programming, but also because Haskell has this aura of hipster programming that I kind of enjoy.
When your industry is growing at a fast pace, it’s a good idea to stay on the edge of things.
I’ve attempted to collect together a bunch of Haskell and post them here for my own benefit, but also you can join with me as we throw off the shackles of OOP and welcome our brave new world of functional development.
Intros and Motivations
Functional Programming should be your #1 priority for 2015 by Ju Gonçalves.
A very good brief introduction to what functional programming is. This article argues that functional programming is the next big thing in computing thanks to distributed architectures (better known as The Cloud). Obviously, this article was written nearly a year ago, hence the title. The collapse of OOP didn’t occur in 2015 and it probably won’t happen anytime soon. Still, this was one of the first articles I came across that motivated me to take functional programming seriously.
This quote sums up the whole article nicely:
That promised time when we’d have applications running distributed and concurrently finally has come. Unfortunately, we are not ready: our “current” (i.e., most used) model for concurrency and parallelism, even though might solve the problem, it adds a lot of complexity.
The Dao of Immutability by Eric Elliott.
A fun meditation on the simplicity of functional programming.
All Evidence Points to OOP Being Bullshit by John Barker
Another cathartic rant on object oriented programming. There seems to be a common theme here…
10 Interview Questions every JavaScript Developer Should Know by Eric Elliott.
Wait, a JavaScript article? In your post about Haskell? Whaaaat? Really this is a functional programming rant in disguise. We all know that JavaScript has a bunch of cruft, but in this article, Elliot argues that one of the most damaging aspects of the language is the class.
I advise people to hire based on whether or not a developer believes in class inheritance. Why? Because people who love it are obstinately stubborn about it. They will go to their graves clutching to it … I won’t hire them. You shouldn’t, either. I’ve seen classes wreak havoc on projects, companies, and lives.
According to the author, that’s not hyperbole. Elliott advocates that the best way to develop JavaScript is to adopt some of the functional traits of langues like Haskell. Being a full-time JavaScript developer, this was the first article that caused me to sit up and take note of functional programming.
Why Haskell Matters from the Haskell.org Wiki
Although it’s fairly dense, this is reasoning for Haskell from the people behind it.
Books
Unless otherwise noted, all these books are freely available online.
Learn You a Haskell for Great Good! by Miran Lipovača
This is the definitive beginner’s guide to Haskell. I remember picking this book up at a Barnes and Noble and thinking what the hell is going on with this language? Years later I actually read the book and I enjoyed the nice pace and quirky illustrations.
Real World Haskell by Bryan O’Sullivan, Don Stewart, and John Goerzen
Haskell is often unfairly categorized as an “academic” language that should remain confined within the realm of computer science theory. This book is both a tutorial and counter-argument to prove Haskell can be used in software development for serious development.
The Structure and Interpretation of Computer Programs by Harold Abelson and Gerald Jay Sussman with Julie Sussman
To be honest, I haven’t read this text. Also, it’s not actually about Haskell, but rather the overall concept of programming language design. It also makes use of Lisp instead of Haskell. I’m only including it here because it’s a monumental work that is often recommended when learning about functional programming.
Videos
Make Haste: Fast Track to Functional Thinking by Katie Miller at CampJS
This is a very fun talk given at a JavaScript conference. It includes a brief overview of functional programming, Haskell, and a framework called Haste used for building web applications in 100% Haskell.
Interesting Projects
Purescript
A Haskell-inspired language that compiles to JavaScript. The language itself is implemented in Haskell, of course.
Haxl
A Facebook open source project for simplifying remote data access.
PostgREST
Arguably the most popular Haskell project around for now. This is a web application that gives your PostgreSQL database a REST API. It’s written in Haskell. I find this to be an interesting project since it is something that most people wouldn’t consider using Haskell for.
Hakyll
Everyone knows about Jekyll, the Ruby static site generator. Hakyll is a Haskell implemented static site generator.
20 Jul 2015
My day job involves doing a lot of GIS work (that’s Geospatial Information Science, for you non-map nerds). Along the way, I’ve collected a lot of interesting data sources for building maps.
I’ve decided to organize them all here. I believe that maps tell a story and by providing tools to build good maps, we can tell better stories.
Some of these links may be broken in the future or the websites undermaintained. If you want to submit a website, get in contact with me.
Data.gov
The U.S. Government has a ton of geospatial data. The biggest challenge is finding it. Fortunately, the recent development of Data.gov has made it slightly easier. Data.gov is supposed to be the one-stop location for all public data, however there are still a lot of individual repositories of data out there that aren’t feeding into it.
Be sure you select the geospatial tag when filtering search results.
Links:
NASA
You already know who NASA is. They operate a very nice website called NEO (for NASA Earth Observations). In particular, the true color 1-day imagery is pretty amazing. They also run a separate website called EarthData which has even more imagery.
VIIRS 1-day for July 17, 2015
Links:
US Department of State
A lot of the data from individual branches of the federal government gets syndicated to Data.gov, but often they get lots in the mix. The Department of State now publishes to the State GeoNode server.
Syria Internally Displaced Person (IDP) Sites, April 16, 2015.
Links:
United States Geological Survey
The United States Geological Survey also warehouses a ton of data and makes it available through their Earth Resources Observation and Science (EROS) webpage. It requires registration to download the data and I’m not sure if it gets syndicated to Data.gov.
Links:
U.S. State Governments
Ideally, all state GIS data should be going to Data.gov, but it can usually be easier to get the data directly from the state government’s website. MIT has collected the web site for every single state GIS group to help you cut down on your searches.
Links:
While they have a boring name, the NSGIC has a really cool database called the GIS Inventory.
Links:
United Arab Emirates
The UAE has spent a lot of effort to build out geospatial collections of the country, which results in some breathtaking imagery of this nation. I suppose if you’re building the world’s tallest skyscrapers and creating artificial islands, you want to show them off.
The Abu Dhabi Geospatial Portal is a good starting point to explore some of this data. Despite the name, it has a lot of imagery for all of the UAE including Dubai.
Ferrari World at Abu Dhabi, UAE
Links:
OpenTopography
Funded by the National Science Foundation, OpenTopograhy is a website that allows you to search through a lot of topography data. They have a very nice interface for finding terrain data and then processing it into one download.
Links:
DigitalGlobe
Being a commercial company, DigitalGlobe doesn’t just give away all their imagery. They do have a few good samples of large cities, plus a gallery of their work.
Links:
NOAA
The National Oceanic and Atmospheric Administration (NOAA) is a government agency that deals with weather, the ocean, and the earth in general. It’s not surprising that they have a good amount of free GIS sources. One of their nicer sources is nowCOAST, a group of services for providing near real-time observations and weather watches and warnings. If you’re a weather nerd, this is some awesome data. They provide a map viewer, but the raw feeds are more interesting.
NOAA Near Real-Time Observations service weather mosaic overlaid on the Mississippi River
Links:
13 Jul 2015
I do a lot of work with geospatial data at my day job. Occasionally, I’ll find a cool data set and I’ll save it off so I can spend some time looking at it and seeing if I can find anything particularly interesting. I found an ArcGIS server hosted by the United Arab Emirates that hosts some very nice imagery of the country.
The imagery is part of the Abu Dhabi Geospatial Portal. Despite the name, the project actually has imagery from all over the UAE, including Dubai.
According to the metadata, this imagery comes from WorldView-2. You can click any of the images to get the full size version.
Here’s a wide shot of the city of Abu Dhabi. In case you’re wondering, Abu Dhabi is not the same place as Dubai. It’s southwest of Dubai and is the country’s second biggest city and capital. This is a beautiful shot of the city meeting the Persian Gulf. But wait… what’s that red triangle in the middle?
Zoom in!
Is that a Ferarri logo?
This is Ferrari World Abu Dhabi. The world’s largest indoor theme park and currently the only Ferrari theme park. There appears to be a racetrack nearby for visitors to test drive a high-end sports car.
Moving to Dubai in the northeast, we can see the famous man-made island archipelago, The World. It doesn’t seem to be doing very well. In addition to being underdeveloped, the islands appear to be sinking into the ocean.
At least one person seems to have made an investment in a private island in the North America section.
There are no roads connecting this to the mainland. The only way on or off this island is by boat or helicopter (note the helipad on the western part of the island). I guess if you like privacy, then this is worth it.
The Palm Jumeirah appears to be faring much better. This is one of two palm-shaped island formations (the other being the Palm Jebel Ali). A third one was planned, but has not been developed.
A detailed look at the spine of the Palm. It’s mostly developed, although you can see one of the fronds is still empty.
The most expensive houses are on the tips of the fronds. This one, on the eastern half, is particularly impressive. While the outer ring of islands is made up of hotels, the inner fronds are private residences.
The canal work in downtown Dubai is impressive. This image also has the most famous landmark in the entire UAE…
A more detailed look at the Burj Khalifa, the tallest man-made structure in the world.
I honestly don’t know what this is; some kind of racetrack perhaps. It has smaller concentric tracks within it.
Some agriculture imagery. As you can image, there’s not a lot of this in the desert. Center pivot irrigation, the style of farming here, makes for those circles which are fun to look at.
A very wide image of a highway outside of Dubai. I like this because it shows they very carefully planted trees around the edges of the highway. It’s probably to keep sand from blowing on the road, but I’m guessing it also creates an illusion that you’re not actually driving through the desert.
When planning neighborhoods and infrastructure, these UAE engineers really like geometric patterns, particularly concentric circles.
More ornate geometric patterns, this time in the form of a highway interchange.
I don’t know what this is. It appears to be some circular roads with some well placed trees extending outwards, but I can’t tell if this is supposed to be a neighborhood development. I think the engineers are just messing with me at this point.
This looks like some kind of resort hotel.
These are some very big yachts. They don’t appear very large until you compare them to the size of the cars parked nearby.
I would occasionally find these compounds that are fenced off. I didn’t look up what they were.
There was a lot of other stuff in the data set. I might return to it and try to find some more crazy stuff. If you’re into this kind of thing, then you should consider learning about geographic information systems (GIS).
Imagery Credit: Abu Dhabi Geospatial Portal
06 Jul 2015
Your first AngularJS app is going to suck.
And so will your second. And third. And probably every single one until you finally understand the framework at an intimate level. Angular is challenging; it will take weeks of development just to learn the basics and years of building non-trivial applications to actually become comfortable with it. There’s a joke that everyone loves to use Angular for new applications, but no one wants to maintain them. Because Angular is so new, I feel like a lot of Angular codebases are people’s first projects. And we’ve already established those are going to suck.
Over the past few years, I’ve been compiling a list of tips and tricks I’ve painfully learned when writing AngularJS code. My hope is that one day I’ll be able to hop in a time machine and slap my younger self with this before I unleash more terrible AngularJS code to humanity. Even if that never happens, you too can benefit from it!
Oh good, they found someone to maintain your first AngularJS codebase.
Keep controllers lightweight
The most common newbie mistake I run across is seeing controllers that contain thousands of lines of code. Most AngularJS tutorials start off by explaining how a controller allow you to put variables in scope and link them to your view, so the new developer begins thinking “okay, this must be where I put all my application logic”.
There’s a few problems with this approach:
- Controllers are not as re-usable as you might think. They typically have only one place to live in the application’s architecture and they are fairly tightly coupled to that.
- Controllers are slightly more difficult to unit test and they usually involve mocking out the
$scope
.
- Controllers can be nested which means the executing environment might change; the downside is now your $scope is different and you could be looking at re-writing a lot of those aforementioned unit tests.
The solution is pretty simple: just use the service and factory AngularJS components and inject them into your controllers. They’re designed for this kind of thing. A well-written factory or service is very easy to unit test and is far more re-usable than controller code.
Here’s a good example. Let’s say you have a really basic web app that grabs a blog feed (as JSON), filters it down by date (passed via $routeParams
), and renders it to a web page.
Bad version:
angular.module('blog-app', [])
.controller('BlogCtrl', function($scope, $http, $routeParams) {
// Grab the start and end date from the route
var start = $routeParams.start,
end = $routeParams.end,
range = moment().range(start, end);
$http.get('/entries').then(function(entries) {
entries.forEach(function(entry) {
if (range.contains(entry.date)) {
$scope.entries.push(entry);
}
});
});
});
In this example, the logic for grabbing the entries and filtering them is tightly coupled to the controller, so we’re not really building reusable code. In addition, the code is tightly coupled to both a hard coded endpoint URL string and the route parameters. For small trivial apps, this isn’t a big deal, but as your web apps grow larger, this becomes awful to maintain. As a general rule of thumb, I never inject $http into controllers. Instead, bundle all of that interaction up into a factory:
Better version:
angular.module('blog-app', [])
.factory('EntryEndpoint', function($http) {
return function(endpointURL) {
this.getEntries = function(start, end) {
var range = moment().range(start, end);
var defer = $q.defer();
var result = [];
$http.get('/entries').then(function(entries) {
entries = entries.map(function(entry) {
return range.contains(entry.date);
});
defer.resolve(result);
});
return defer.promise;
};
};
})
.controller('BlogCtrl', function($scope, $routeParams, EntryEndpoint) {
var start = $routeParams.start,
end = $routeParams.end;
var endpoint = new EntryEndpoint('/entries');
endpoint.getEntries(start, end).then(function(entries) {
$scope.entries = entries;
});
});
This “better version” is a lot longer and appears more complicated, but the payoff is that we now have a clean separation between our components; the controller hands off the logic to factory which doesn’t care about a lot of the other things (like the route parameters or the current scope). We can also re-use this code in other controllers in other places.
Keep the logic out of your templates
One of the reasons I despise PHP is that it encourages you to mix your logic with your HTML code. This is basically just setting you up to write big monolithic and unmaintainable applications. AngularJS’s templating functionality, while being very good, can also lead you to load up your templates with a lot of logic that really doesn’t belong there. The ngIf directive in particular is a beast because if you rely on it too much, you might just have a huge chain of ngIf directives in your templates, which isn’t clean or fun to maintain. Angular also lets you manipulate the scope variables (or create new ones) from within the template code which is also something I’m not a fan of.
Here’s some template code that I’m used to seeing…
Bad version:
<ul>
<li ng-repeat="entry in entries">
<div ng-if="entry.attribute === 'valueOne'"> <!-- do something --> </div>
<div ng-if="entry.attribute === 'valueTwo'"> <!-- do something else --> </div>
<div ng-if="entry.attribute === 'valueThree'"> <!-- do yet another thing --> </div>
</li>
</ul>
This is a really simple case, so it doesn’ts look too bad, but if you continue adding logic here, you’ll eventually get into nested ng-if statements or complicated boolean expressions that could otherwise be elegantly expressed in your controller code.
So how do you avoid junking up your templates with logic? I have a few methods I like:
- If you find yourself using a lot of logic to build out a view stack as your application state changes, then you probably should be using
ngRoutes
to cleanly define how your view changes between application states.
- Define your JavaScript objects to adhere to an interface and keep your view code from needing to know the implementation; this is just good development anyway.
- Angular directives allow you to couple together your logic and view together (when it makes sense to) into a maintainable package. When you use directives with
ngTransclude
, you end up with a very powerful way to write clean code that is also highly flexible. More on that later.
Favor Factories over Services
This may be more of a personal preference than a hard rule, but I think Factories are underused when it compared to Services. The big difference between the two is that Factories can return a constructor definition which can be instantiated with new. We can, of course, pass parameters into the constructor, to custom tailor the object to the current situation, rather than just writing a giant service that tries to be one-size-fits-all.
This isn’t a 100% of the time rule (since Services are nice for simple interactions), but it’s really clean in the long run when you’re trying to build out your app’s domain objects.
Mixing AngularJS and jQuery
Here’s a fun piece of trivia: AngularJS includes a copy of jqLite and it will bind it to the global $
variable. If you include jQuery before you include the AngularJS library, it will use that instead. Because jqLite isn’t completely compatible with jQuery, I make sure that I include jQuery first so it doesn’t have any adverse effects on other plugins.
While on the topic of jQuery, it should be noted that AngularJS and jQuery aren’t incompatible libraries, however I really don’t like mixing jQuery DOM manipulation code with my AngularJS code. I see a ton of this in AngularJS code written by people who are new to the framework because they have exerience with jQuery and they used to solving all their problems with it. DOM manipulation within your Angular code tends to be bad since it couples your DOM view code to your logic.
There is one good exception, however: Angular directives give you access to the DOM element in the link function. Go crazy with DOM manipulation there. Other than that, I avoid it. If you absolutely must do some jQuery magic to manipulate the DOM, at least restrict it to controller code, which already has a tighter coupling to the view code anyway.
One module per file.
Prepare your hate mail! I am not a fan of splitting a module up into multiple files. Most novice AngularJS developers don’t even know this is possible, but it’s perfectly fine to have a file called first.js
with:
angular.module('my-module', []).controller('MyController', function() {});
and another file called second.js
with:
angular.module('my-module').controller('AnotherController', function() { });
A lot of people probably like this; it keeps your files small and normally that’s a good thing. The nastiness comes from the fact that you need to be absolutely sure that the first.js
file is loaded before the second.js
file, otherwise you’ll get a dreaded $injector:nomod
error. You see, that first call to angular.module has two arguments, so we’re defining the module there. The second call just allows us to continue the module. If you’re including your .js
files in a page, it’s not too bad to make sure this happens, but if you’re using something like grunt, you need to be sure your modules are being concat’d in the right order (the default is alphabetical). Some people get around this by creating “meta-modules” that just exist as dependency holders. This isn’t terrible, but I think it’s just going to bloat up your project structure. My personal preference is to keep things as one module per file.
Use ngRoutes
Someone once told me they didn’t use routes because it made their URLs look ugly. A small part of me died that day.
Routes are incredibly useful for building out your application states. I like to design my routes as early as possible in the application design since it’s much tougher to add them later. Even though the ngRoutes module isn’t part of the core AngularJS library, you should be using it.
Also, if you’re still hung up on ugly URLs, you can always switch it to HTML5 mode or opt for the Angular-UI-Route 3rd party library.
Don’t inject $routeParams
…into anything other than controllers. It’s really tempting to just inject that thing anywhere you need it, but injecting that provider into, say, a factory or a service, just tightly couples that component to your specific route.
Inject $routeParams
into your controller code and have that pass the route variables to your individual objects in other ways (like in a Factory constructor, because those are a good idea, right?).
Don’t manipulate things in $rootScope
This is fairly obvious. The $rootScope
is basically a global variable space that gets inherited for all child scopes. Putting things in the $rootScope
isn’t too terrible (as long as you’re only doing it in one centralized place), but changing those variables will now change them for all scopes. You’re basically dealing with global variables at this point.
Nested Controllers make things more complicated
Being able to nest controllers is pretty damn cool. I’m okay with it. In fact, you probably can’t write a modern web application without doing it. Just realize that doing this is going to increase the complexity of your application. If you’re nesting controllers four or five levels deep, then you probably need to think about your web app’s architecture.
Nested controllers tend to be coupled together, which is bad for all the obvious reasons.
Try to avoid using $scope.$parent
A newbie mistake that I see is trying to manipulate a variable in the current scope without realizing that things like ngRepeat
create a new child scope. The less-than-ideal solution is to manipulate it with $scope.$parent
. This is not an elegant solution, particularly if this scope is more deeply nested (then you get things like $scope.$parent.$parent.$parent
).
My preferred solution is to wrap these variables in an outer object so that the child scope inherits a copy of that object which has the pointer to the actual variable. This is not the most intuitive thing to understand, but it’s far more maintainable than a family tree of parent scopes.
Directives are really cool
They are also incredibly complicated and difficult to learn, but if you can master them, you basically can have a deeper appreciation for how AngularJS works. One aspect of directives is particularly cool: ngTransclude
. While directives are usually self-contained and tightly encapsulated components of your application, transclusion allows you to elegantly add on to the directive to tailor it to your specific use case. It’s too detailed to try to explain here, but it’s seriously a pretty cool way to build some sweet directives.
Conclusions
AngularJS is a beast. The learning curve on this library is insane. I feel that by learning it, however, you will become a better developer and you’ll curate your own lists like this. Hopefully you can pass your own experiences along to others.
Photo Credit: Landfill, by the Wisconsin Department of Natural Resources.
30 Jun 2015
This is part two of a series where I wanted to write about how doing small things when setting up your project can have surprisingly positive effects.
This material was originally discussed at the NOVA-Python meetup by organizer, Ryan Day. His original focus was on developing good project layouts for Python + Flask web applications, but I think the material works well in a variety of environments.
Learning through Play
In high school, I wasn’t great at math. Listening to a math lecture was just boring. On top of that, my dad was a math teacher so I’d get math lectures at home as well. I think math is inherently a really bad subject to teach through the standard lecture format; it’s unfortunate that it’s still being taught that way. So while math lectures couldn’t keep my attention, I had a really dorky fascination with plugging things into the ol’ TI-83 graphing calculator and seeing what would come out. As the teacher was trying to drill into our heads what y=mx^2+b
meant and how the quadratic formula could solve that, I thought it was far more interesting to just plug that into the graphing calculator and watch the points cascade over that small LCD screen. Then I’d tweak the variables around just to see what would happen.
Yeah, this is really nerdy. It’s no surprise I eventually decided to study computer science.
What I had no clue I was doing was actually learning in a much better way. Instead of just learning a formula, I was interacting with it; in a more whimsical version, I was playing with it. It’s only been recently discovered that this is not only a legitimate way to learn, it’s also really effective, particularly when it comes to software development (Sonmez).
Making your Project Friendly for Interactive Learning
This part’s incredibly easy, and yet not too obvious. When we join a project and check out the code for the first time, we tend to see a directory structure that has the following things:
- The source code itself
- Supporting documentation
- Configuration files
- Unit tests
What’s missing is a place for the developers to play around. Now, any developer can create this, but it’s a nice psychological boost to communicate that it’s okay to just play with the project and learn about it outside of running unit tests or wherever the program’s main entry point starts.
To make this work, just create an empty directory and name it whatever works. For Python projects, I like the name user-scripts. For Node.js projects, I like user-bin
. Now toss a README.md
file in that directory that tells the developer “Hey, this directory is for you. Go crazy. Write a bunch of weird one-off code that will help you learn about the project. Nothing in here will be saved to version control.”
Now the important part is to check that directory into your version control, and then immediately add it to your .gitignore
.
app
- scripts
- styles
- libs
- docs
- test
- user-scripts
Now your new developers have a place where they can write code, play with your project’s internal libraries, and just get a feel for things through interaction.
Yeah, It’s not a big change and it’s not exactly profound or novel. Maybe your new developers are already doing this on their own, but in my experience a lot of new developers don’t have the instinct to try this out. There may be an implicit fear of breaking something or accidentally checking in something that will make them look dumb in front of the more experienced developers. They may feel that they shouldn’t be spending a lot of time writing code that isn’t going to make it into production. If you can break this perception, then you can really make joining a new project easier and more fun.
Bibliography
Sonmez, John. Z. Soft Skills: The Software Developer’s Life Manual. Manning Publications, 2015. 223. Print.