The Highgroove blog. Sit pit-side with us to learn how we work. Sometimes technical, sometimes business-oriented, but always focused on simple solutions.
You are browsing posts about Open Source. Check out all posts on our blog.
by gregg
Ruby ships with an Interactive Ruby Shell (IRB) that every Ruby developer has played around with at some point. It's a great place to experiment with unfamiliar methods and running bits of code, but if you really want to dig into an object or do hardcore debugging, it can leave a lot to be desired.
Pry is a great IRB alternative that has a number of features that make it one of my must-have tools.
→ Read More
by chris
Highgroove requires everyone to attend a conference every year, and begrudgingly I agreed to head to Hawaii for Aloha Ruby. The flight was long but well worth it, the weather was amazing, and the conference was packed with useful information.
→ Read More
Highgroovers take test-driven design (TDD) seriously, but it's easy to become overwhelmed by the number of tools available for testing applications. However, if you know us even a little, you know that we have a lot of experience using different tools—and that we've developed some opinions about them.
To help others narrow down the options, I asked a couple of developers here how they felt about the two most popular tools for acceptance-level testing: RSpec and Cucumber.
→ Read More
At Highgroove, we love working with the community, whether it's hosting hack nights, organizing ATLRUG, or contributing to open-source software. One of our key contributions involves auditing those ruby gems upon which we most rely, or the ones that present the most perceived risk. Ernie Miller's Squeel definitely piqued our security interest. Read on to hear about the results of our audit and how we worked with Mr. Miller to validate our results.
→ Read More
by jonathan
A long time has passed since we've last spoken about processing background jobs. Much has changed regarding the tools for asynchronously processing long running tasks in Ruby and Rails. Most recently we've favored Resque, especially now that Heroku's cedar stack supports it.
There's one problem with Resque. Enforcing strictly serial job semantics is impossible in Resque without custom development or limiting the number of workers.
So, NEVER use resque for serial jobs, OR read on to find how we resolved this dilemma with resque-lonely_job gem, a new resque plugin.
→ Read More
For those following along at home, you know that we love to work iteratively, and as such, work towards our big-picture goals one small step at a time. We also take software quality seriously, and an important part of software quality, particularly when it comes to web applictations, is security. We've begun to develop a new process to address security in our applications. In the process we also want to contribute back to the community where we can because it will make the entire web ecosystem that much better. After the break we'll talk about our high-level process and how we implement it.
→ Read More
by gregg
Often times the state of an object is tracked through boolean attributes, but
this solution doesn't scale well since each new state requires a new column in
the table along with logic to govern how the states change. If you have a
model which you need to track the states of, the
state_machine gem can be a
great solution. It stores the state of the object in a string-type column
(state in the example below), and adding states and transitions is as easy as
adding a couple lines of code to the model.
by david
At Highgroove we love giving presentations. We sponsor the Atlanta Ruby Users Group (ATLRUG), we each attend at least one conference per year where we are highly encouraged to speak rather than simply attend, and we give weekly tech talks at the office. Each Highgroover has their own presentation tool-stack ranging from extremely simple using nothing more than a web browser with various tabs they manually rotate through, to sophisticated Keynote slide-decks. One can also use a very editor and CLI-centric collection of tools to build flexible presentations using many of our everyday tools like git, guard, slim, and Github with Github Pages. If you like using those sorts of tools to speed things along read on after the jump.
→ Read More
Highgroove really likes Pry. It's a great tool for digging into your code and seeing what's going on with tons of great features. However, there are situations where using a standard binding.pry breakpoint will not block your program and allow you to inspect it. I recently ran into this situation when trying to debug an application that used Foreman to manage it's processes. Luckily, the pry-remote project turned out to be a great solution.
by gregg
I'm just a week into my first real job and I feel like the luckiest person in the world. Many people fresh out of graduate school still have to "pay their dues" at a company that values being in your cube by 8:00 am even if you were up until 1 last night getting another report finished. While graduate school is essentially a ROWE, there are still expectations to be in a chair from 9:00 - 5:30 (at least in my lab there were). While having the freedom to work when you want and how you want is amazing, it doesn't come without growing pains. After a couple of years of getting judged based on time in the lab, it's hard to remember that it's what I get done that matters and not when I do it. Read through to see how I managed to commit what would be a serious faux pas in any workplace other than a ROWE: I took a day off in my first week.
→ Read More
As the web continues to mature, JSON APIs (and XML if you're into that) have become increasingly important. But if you've tried to use Rails to write an API recently, you know there are a handful of competing methods and gems focused on making this better. I'm all for interchangable libraries, but, as Yehuda Katz pointed out in his recent talk at RailsConf, Rails needs a "convention over configuration" approach to solving the JSON serialization problem once and for all. So I was pretty excited when I heard about Jose Valim's ActiveModel::Serializer.
→ Read More
by mike
Highgroove sent me, Jonathan, and Patrick to RailsConf 2012. I've been learning a lot by going to some great talks and enjoying talking to my fellow Rails developers. If you see any of us, feel free to say hello!
There's a big push to make this the most public RailsConf ever, so all talks are being recorded. On top of that, I have also been trying to take notes and make them public.
→ Read More
Sunspot, a Ruby interface for Solr
usually backed by ActiveRecord objects, recently introduced native geospatial
searching capability in the master branch (currently a 2.0.0 prerelease).
While the functionality has not been released as a stable version yet, I felt that a blog post introducing the features is appropriate with the hope that folks will try it out and give feedback!
Read on for a short example.
→ Read More
Database-bound tests are a drag. Inconsistent tests are a pain. Database-bound, inconsistently failing tests are the worst!
The following commit message is from a real code base:
Run in transactions by default.
When we added controller specs they weren't being run w/any kind of DB cleaner b/c there was no default strategy and they weren't explicitly included in a group. Now, we use :transactions be default, setting request specs to use :truncation
Also, I saw a 2 second speed up from this change!
Let's look at what we changed in this commit to turn our inconsistently failing database-bound tests into slightly faster, consistent, database-bound tests.
→ Read More
by mike
I started watching Gary Bernhardt's Destroy All Software screencasts recently and after watching a specific episode, I had to have his Ruby testing setup. After sitting in Vim config for a while, with some improvements I made, I started feeling like I should somehow contribute my changes back. After I started adding a few more changes suggested by fellow Taconaut Steven Harman, I decided it really needed to be a Vim Plugin.
→ Read More
This weekend we Highgroovers (a.k.a. Taconauts) took some time to do one of the things we really love: create and release Open Source Software. In fact, we released not one, but three new tools into the world: grocer, git_tracker, and puppet-osx_defaults.
→ Read More
by pamela
Last night at the Highgroove Studios office, we held the March edition of Hack Night, our monthly social coding gathering. We focused on starting, polishing, and/or discussing open source projects ranging from the super useful to the super silly (beer is always provided at our hack nights...)
→ Read More
by stafford
Last month I had the privilege of attending the very first SpreeConf in New York City. If you aren't familiar with Spree, it is an awesome Rails e-commerce engine you can use to build a full-featured online store. The conference was held over two days; the first day featured several training sessions. The sessions covered a range of topics including theming, configuring, and testing Spree. The second day was filled not only with talks related to Spree, but to e-commerce and Open Source in general.
→ Read More
by aubrey
Prior to working at Highgroove, I worked for a number of softare product companies. One of the best metrics I've found for measuring how great a company is to work for as a developer is the Joel Test, which is a list of 12 simple questions. I've worked at companies that have scored high and companies that have scored low, and please believe me when I can tell you that the issues on that list really do make a difference.
My only issue with the Joel Test is that it is really geared toward software product companies. While a lot of the ideals translate to a software consulting company, the specific questions don't always apply when you are working on a lot of projects for a number of clients. So, in an effort the help measure what makes great software consulting teams so great I present...
→ Read More
At Highgroove, we're always looking for new ruby gems to help speed up development and keep the code DRY. The gem I found this time was Heroku's very own Databasedotcom gem, a fantastic Salesforce.com API wrapper for Ruby! I must admit, I wasn't very surprised to find a gem since Salesforce owns Heroku, but I did not expect it to be so fantastic and easy to use. Here's how!
→ Read More
by stafford
One of my least favorite chores as a developer is dealing with email. I’m not talking about my inbox. That is a post for another day ;). I’m talking about emails sent by web applications. Whether it is a sign up confirmation email, a receipt from a purchase, or reminder for your dog’s birthday. Chances are, if you have a web application, it sends email.
Traditionally, my workflow for testing these emails has not been very elegant or even efficient. It would either involve creating a bunch of users with different emails accounts I own, or telling the back-end to send all emails to my email address. While both of these work to some extent, the former is very time consuming and the later isn’t really testing the system the way it is meant to be used.
Mailcatcher one-ups both of these methods big time. Mailcatcher provides you with a local SMTP server for you to send your emails to in your development environment. Mailcatcher also provides you with a webmail interface to view all the emails your system has sent.
→ Read More
by mike
As a developer, I always try to follow the "Boy Scout Principle" when it comes to the code I'm working with. Simply put:
always leave the code cleaner than when you found it
based on the idea that Boy Scouts always leave a campground cleaner than when they found it.
It is tempting to make sweeping changes and clean up lots of code while implementing a small new feature, however, it's also good practice to separate commits that don't include any unrelated changes. Occasionally this means that a file that has a few small changes that clean up code, and others that are dealing with some functional changes, needs to get committed. Luckily, git has an awesome feature that allows us to get exactly what we want: hunks.
→ Read More
by jonathan
In case you missed it, the awesome Globay Day of Coderetreat occurred on December 3rd. The amount of fun I experienced was unexpected and impressive! I learned some things too. Read on to find out what.
(Also, don't worry if you missed the code retreat, sad kitten has some good news for you at the end of this post.)
→ Read More"With all of the people doing this, why hasn't anyone ever just made a ..."
"I really like doing this, but everytime I get to this point I struggle. I need to make that better."
"Man... what I really need is something that makes this job less tedious!"
Those sentiments have reared their ugly heads in some way or another with every crafty endeavour in which we at Highgroove have gotten involved. Sometimes those crafts are working on bikes (crafty and artisanal as well as mechanical), brewing beer (crafty, artisinal, and after some patience, tasty if you're careful), working on rails apps (crafty and often fun!), or pulling shots of espresso (crafty, tasty, and caffeinated. Sometimes with latte art.) Instead of simply wishing there was a solution, Highgroove encourages us to solve the problem, not just for ourselves, but for others.
→ Read More
Cluster analysis methods have been gaining popularity as a way of Relating pieces of data in large datasets with one another. Examples in social networking are obvious: friends on Facebook cluster into cliques and communities, which cluster into even larger groups. Demographics and other marketing research can also be aided by sorting prospective customers into groups based on preference.
→ Read More
Active Shipping is a nifty Shipping API extension for Active Merchant. It provides methods for interacting with common shipping carrier APIs. Recently we used Active Shipping on a client's e-commerce site to provide customers with many shipping options from several carriers.
→ Read More
by vanstee
I recently got to work on some really interesting, big data problems at Highgroove. One of our clients needed to record every api call and analyze specific time periods for averages and usage metrics. Hadoop fits this use case pretty well with a distributed file system and map reduce framework built in. But, I'll be honest, I wasn't too happy about having to write map reduce jobs in Java. While looking for alternatives I found Wukong; a gem that adds a Ruby wrapper around the Hadoop Streaming utility. Here's an example of how easy it makes writing map reduce jobs.
→ Read More
by drice
Ruby on Rails is an open source, free framework for rapidly prototyping and developing robust, scalable web applications. Ruby on Rails is excellent all by itself, but one of the greatest parts of the Ruby on Rails community is the extensive list of code libraries, plugins, and extensions, often shortened to just: Ruby "gems", hosted on RubyGems. As of writing this, RubyGems reports that it is hosting over 28,000 gems!
That is not a trivial number. That is 28,000 code libraries that you don't have to write to get your application up and running. That is hundreds of thousands of lines of code that have already been battle-tested in production applications for you. That is getting your application developed and ready for market faster and more efficiently, by re-using code and not re-inventing the wheel at every turn.
But there's still a problem with all of these gems... How do you find out if a feature you want already has a gem for it amongst all of the options? Sometimes they have weird names. You might think, "maybe I am just not searching for the right word?" Even Highgroovers have a tough time searching through this treasure trove of code, so we keep a list of Gems that solve common problems well.
Here are some of our favorites in use on many of our current client's projects (and our own projects):
→ Read More
by brian
Bayesian networks have proven extremely useful for classifying events and documents, reliability analysis, and in many other fields. Essentially, wherever a well-defined chain of causation given between many pieces of data exists, a Bayes net can help provide probabilities for the "hidden variables" of a system: in the cases above, for example, the category a document belongs to, the probability a system will fail if a certain component fails, etc.
As part of another project here at Highgroove, I've been developing a gem called glymour that learns a Bayes net's structure automatically, which is important when it becomes impractical to manually define causal relationships (e.g. when taking into account dozens of different variables). Working on an open-source gem while relating it to a larger project has made me understand one of its many benefits: open source code is kind of like a constrained writing in which you are constrained to being as general-purpose as possible (and reasonable).
→ Read More
by vanstee
If you've ever tried to integrate Jabber into your app or just wanted to write a bot to respond with TWSS jokes, you've probably checked out XMPP4R or Switchboard. These libraries are great and have a really solid feature set, but if you find yourself writing a lot of boilerplate and really want a library with more of a Ruby feel to it, try out sprsquish's gem, Blather.
It's written as a DSL and leverages Eventmachine, which allows you to easily listen for messages and send a response. Checkout how simple it is to get up and running:
→ Read More
by chris
Feature Flags are one of my favorite patterns. Ross at Flickr blogged a pretty good description on the Flickr Code blog in 2009, and a Ruby gem ‘rails-settings’ appeared in 2009 to give Rails developers a “Global Hash” which can be used to implement feature flags. This has been forked by a handful of people, updated to work with Rails 3, adding support for settings associated with models, etc.
However, the official rails-settings gem hasn’t been updated since 2009 and many of the forks are not of an acceptable level of quality for us to use. Some have removed tests, some have lots of backwards compatability code, and most are pretty hard to read. I spoke with some other developers and decided to build a new one from scratch, using rails-settings as the inspiration, so here we are!
by skalnik
Here at Highgroove, we do code reviews, which ensures everyone is reading some code on a regular basis. However, being a developer usually means you'll be reading code regularly anyway. In the past, I used to avoid this if I could. However, now I've learned how much I can get from it, and use it to improve myself as a developer.
Reading code isn't easy, and I certainly didn't want to do it when my only goal was figuring out why a gem wasn't doing what it promised. It was much easier to just find someone familiar with it and pick their brain. However, there's more to be gained from reading source code than you may think.
→ Read More
by andy
At Highgroove, we build database-backed web applications. These days, there are many options when choosing a database backend. We normally start with relational database systems (e.g., PostgreSQL and MySQL) because they are very mature and feature niceties like ACID transactions and locks.
On the other hand, we also work with newer NoSQL database systems that often trade features like transactions, locks, and joins for higher performance and scalability. One popular option is a document-based store called MongoDB. By design, MongoDB does not support ACID transactions (though many operations are atomic) or traditional locks.
In many cases MongoDB, paired with an object-relational mapper like mongo_mapper is a great solution for Ruby and Rails applications. And after the break I explore a solution that allows a developer to “lock” a MongoDB document while still maintaining the high performance and other features MongoDB is known for.
At Highgroove, we test our code extensively. The best testing code has solid coverage, yet also shares many of the qualities of well-written code in general, such as readability and lack of repetition.
The popular testing framework RSpec ships with constructs that aid in writing elegant testing code. One advanced technique to keep things DRY is shown after the break.
by chris
When building fancy AJAXy apps that have complex user flows outside of page navigation, it’s important to let users know if they are about to trash something they just wrote. Ever had a browser lose several paragraphs of text you typed in? Not fun!
This can be done as complexly as you want, but at Highgroove we’re big fans of KISS so here’s a simple approach to making this happen. It assumes you’re using jQuery, but the same technique should work with any other JavaScript framework.
by jonathan
He may be too humble* to admit it but Andy Lindeman, one of our new(er) hires, is awesome.
He recently did a lightning presentation on Solr and Sunspot for the Huntsville Ruby Group. See the announcement over on his blog and give him some feedback!
* We only hire the humblest programmers. It is in the company charter.
by mtodd
April 9th and 10th in San Francisco was the first ever CodeConf. Read on for a recap on this incredibly insightful and wonderful conference.
Virtually every application we push to production relies very heavily on open source code in one way or another. This means we’re utilizing the experience, knowledge, and expertise of the OSS (Open Source Software) community to solve many of our problems. Things that would’ve taken us quite a bit of time, money, and effort to iterate on in order to bring the quality to something on par with the OSS options are freely available and open for us to incorporate in our projects.
by chris
Bug-free code is hard to come by, but we strive to do the best we can. The most important part of this is automated testing, and code coverage metrics are a commonly used indicator of the quality of tests. However, code coverage is a misleading metric: Having “Good” code coverge in your tests isn’t really an indicator of good testing, but having “Bad” code coverage definitely means that your tests are insufficient.
In Ruby (and most other common languages), it’s possible to get “C0” code coverage metrics. All this means is that a particular line of code was executed. It seems like this would be the holy grail of testing, but it’s anything but that. Heres a simple psuedo-ruby example of something that would not be adequately checked even with 100% C0 coverage:
def simple_check? true end def my_great_function simple_check? or (foo? and bar? or (baz? not bat?)) end test "my_great_function should work" do assert_equal true, my_great_function end
The foo/bar/baz/bat logic never gets looked at in the test. While a contrived example, situations like this come up a lot in code and just because one line gets executed, there could be all sorts of bugs.
A more appropriate use of C0 metrics is to identify code that is “definitely” undertested. If you’re tasked with getting some legacy code that is new to you and isn’t very well tested into shape, running C0 metrics will give you the places you should start looking first.
Other types of coverage include:
To get C0 metrics, we used and loved a little Gem called rails_code_qa with our Ruby 1.8 applications. It runs a handful of tools including rcov which provides C0 code coverage information, but due to changes in Ruby rcov no longer works with Ruby 1.9.
Enter cover_me. This gem gives you the same C0 sort of metrics that rcov does, but it works with Ruby 1.9. Setting it up is very straightforward, and the end result is a simple report (that automatically opens in your browser!) that provides C0 information both in aggregate and for each of your files. cover_me works with all the common Ruby test frameworks, and there isn’t a reason to not include it in all of your projects.
If your C0 coverage is around 100%, your code to test ratio is above 1:1 (We try for 1:1.4, you can get this number for Ruby on Rails projects by running `rake stats`), and you’ve been doing Test Driven Development all along, chances are pretty good that your code might well tested.
You are writing automated tests for all your code, right?
Highgroove does, and if you’d like us to take on a project for you, we’ll deliver uncommonly bug-free code.
by chris
At Highgroove, we’re experts at glue. Not that kind of glue, but the kind that connects one technology to another.
We build on lots of great Open Source tools and libraries to deliver finished products to our customers that do exactly what they want. Without these pieces, we wouldn’t be able to do what we do.
For a project I’m working on right now, we need to be able to post to Twitter and Facebook on a schedule from within the application without requiring the user to push that ‘share’ button. The end result is something that seems like just two clicks in the UI, and isn’t much code, but took some thinking to get it working right.
So here’s what I put together, for you, for free! To those of you that put together the APIs at Facebook and Twitter, and those that worked on the Twitter gem and the OAuth gem, Thanks!
All the details aren’t there, but if you’re familiar with Ruby on Rails and working on a Rails 3 project, the code snippets there should be enough to get you talking to Twitter and Facebook. Let us know if you build anything cool based on those!
by mtodd
It’s late Friday afternoon, I’m angry at this problem I’ve been working on for an hour (or more), but I’m stuck. The sun is out and a cool breeze is blowing softly outside and I want to be riding my bicycle through the streets of Atlanta at full speed! Right now, I can think of nothing more than how much I hate this bug. I’m stumped.
What do I do?
If you’re keen on the tongue in cheek, check out my amazing blog post about integrating facebook into a rails app.
For the short and sweet version, you can jump to the README on my github fork of facebooker2.
by mtodd
Most database-backed applications, especially Ruby applications written in Rails do fine with a SQL Database, like MySQL. Adam Wiggins of Heroku does a great job explaining how SQL Databases are an Overapplied Solution. There are definitely a few cases we’ve seen where a NoSQL solution like Redis can really shine.
At Highgroove, we’ve got several projects utilizing the Redis key-value store. Here are a few reasons you might want to look into Redis:
1) “Data is frequently written, infrequently read” – if you are making tons of writes and MySQL can’t keep up, Redis has been clocked at 110000 SETs (the INSERT equivalent) per second!
2) “Data can be expired” – if you have data that can be expired on a regular basis, over time, or explicitly, like stats, logs, and session data, Redis can explicitly expire whole keys or quickly trim large lists of data in no time.
3) “Data is a collection of COUNTs, or SUMs of other data.” – if you have ever written an UPDATE statement that adds +1 to a record, you probably know that it is quite expensive, and could possibly be wrong. Redis has built in support for incrementing and decrementing values, quickly.
We love SQL Databases, so you won’t see us abandoning them any time soon, but we enjoy adding tools like Redis to our toolbelt.
More Information on Redis:
Redis Project Page – http://code.google.com/p/redis/
Redis (wikipedia entry) – http://en.wikipedia.org/wiki/Redis_
A Collection of Redis Use Cases – http://www.paperplanes.de/2010/2/16/a_collection_of_redis_use_cases.html
by cbq
A new client of ours had a big problem. The site they built was getting too many searches (a very good problem to have). The searches all used Andre’s geokit gem and the geokit-rails plugin to provide local results.
Even with the library’s multi-geocoder support (Google, with failover to Yahoo), the site was hitting the limits imposed by both services every day!
So, we quickly implemented a query caching mechanism that caches geocoding lookups that don’t change very regularly, saving the site from making all those API calls. But, the cool part is, we actually added this functionality to the open-source library itself, and the client’s application now directly benefits.
Several things will now happen, because we contributed back to the open-source library, instead of keeping this “addition” to ourself (and the client):
Truly leveraging open-source technology, when done right, can be a huge win for everyone: the client, vendor, and community.
Trying to handle image manipulation, creating PDFs, or in-memory caching in pure Ruby is like trying to win the Tour de France on your hipster single-speed bike. The single-speed works 90% of the time, but when you have demanding performance requirements, it’s not good enough. Many popular Ruby libraries, such as MySQL/PostgreSQL, RMagick, and most of the webservers Ruby applications are deployed on (like Passenger, Mongrel, and Thin), harness the blazing speed of the C language and libraries to handle the heavy lifting and performance-intensive business that Ruby can’t keep up with on its own.
In some of my recent work, I had the opportunity to delve into and expand on a Ruby extension written in C for looking up geographic information based on IPs. This library was vital to one of our client’s projects that has immense performance requirements without the possibility of full request caching. By utilizing the existing GeoIP C library for accessing their special in-memory binary database, we were able to keep up with the demand the application would be seeing.
As is common at Highgroove Studios, along with making sure our contributions to the library were open sourced, I took the lessons and experience gained from this unusual endeavor and presented them to our local Ruby User Group here in Atlanta. I focused more on exposing the bridge between the Ruby and the C environments and understanding the internals of the Ruby language from a C standpoint. However, armed with this knowledge, any Rubyist is able to open up most any Ruby extension or even the Ruby language implementation itself and understand what’s going on. My goal was to get the developers over the initial hurdle of being able to read the code and understand it enough to investigate further.
Personally, I gained from this experience a better appreciation for the real beauty of the Ruby language and the effort required to make it as fluid and dynamic as it is as well as having a more thorough understanding of the internal workings of the language. Working this close to the language core has also made a difference on my Ruby style, both in trying to fight the language less but to also use it more efficiently and effectively.
For more information, check out the presentation slides1 and some of the C examples I wrote for the presentation2. Also check out the GeoIP I contributed to which inspired this whole adventure3.
1 http://www.slideshare.net/maraby/writing-ruby-extensions
Both James Edward Gray II and Matt Todd were quoted in Satish Talim of Ruby Learning’s Poll: 20+ Rubyists are using Sinatra – Do You?
Sinatra is a Ruby framework for quickly creating web applications with minimal effort — a DSL for the web. We use Sinatra for several client projects, and it is also an integral part of Scout.
by kevin
Check out what’s going on with Scout and Merb in Atlanta this weekend!
by james
I had an unusual Christmas Eve. While my wife was entertaining the family, I was in my office programming until nearly midnight. (Have I mentioned how much I adore my wife?!)
What had me so tied down at such an odd hour? One of the open source libraries Highgroove maintains: FasterCSV.
The Ruby core team gave me the nod to replace the standard CSV library with FasterCSV just before the Ruby 1.9.0 release. While a little more warning would have been nice, I was happy to do it.
I think it’s important to discuss why Highgroove, not just allows, but actually prefers that I spend some of my time maintaining our open source libraries, like FasterCSV.
The reason we created the library is nice and simple: we needed it. We import or export spreadsheet data at some point for most of our applications. We work with large data sets and we needed that to be fast. We also frequently use headers in the CSV files to map directly to database fields so a broader feature set to support such things was important to us.
Now we could have built it and just used it internally, but we decided to share. Did we do it because we are just really nice guys? Not really. (We are though, of course!) We did it because we wanted it to be great.
There are many subtle variations and edge cases for CSV. We debugged the library the best we could, but we’re human and we missed things. That meant our library would have trouble with some inputs. We (hopefully) would have noticed those eventually, but we’re just one set of users and by releasing it on the world we multiplied the user base by thousands. Needless to say early users found problems for us much faster than we could have alone. Better than that, they sometimes even fixed them for us! Even when they don’t, they usually send in an example that leads us straight to the problem which is almost as helpful.
That’s not all. Remember all of those features I told you we love? I wish I could brag and tell you I invented all of them, but some of my favorite features in FasterCSV were sent in by the users. I got the ball rolling with what I could see us needing, but the users ran with the ideas and made my library better than I knew it could be.
Highgroove isn’t losing hours on me applying a few patches here and there. They are gaining the help of many extra employees. We love that.
To top it all off, Matz gives us the ultimate Christmas present. By blessing our work, he saves us the trouble of even installing it for new projects. Very soon we will be able to count on our code being available in all modern Ruby installs. That’s just one less thing we need to worry about.
With open source everybody wins.
If you are following the Capistrano preview releases, you may have noticed a new dependency. Capistrano now depends on HighLine, an open source input library by yours truly.
The reason for the switch is that Capistrano needed a reliable way to grab passwords in a cross-platform way. That turns out to be a lot harder than you might guess. On Unix, termios can make short work of such challenges, but that’s an extra C extension install and it doesn’t work on Windows.
HighLine combines the knowledge of several platform gurus to use the right solutions in the right place. Even with all that knowledge as an advantage Capistrano’s maintainer, Jamis Buck, still had concerns. termios can’t be made a HighLine dependency, since we want to stay cross-platform and when defaulting to stty HighLine was a little flaky for the way Capistrano users might need it. Jamis and I discussed these concerns and HighLine was patched with better support for Capistrano’s needs. Jamis later added the dependency and HighLine benefited from another round of expert knowledge.
It still impresses me how much we can accomplish with the super friendly open source model of development. Thanks for the input Jamis!
