Develop gems with ease using devgem

For my master thesis, I was working on a low-level project where I have to develop a few gems that are not only dependent on one another, but would have to be developed in parallel. One of the main problems I faced was adding the dependency in a proper way that would be clean (not hackish) and DRY.

The problem

The problem arose when I wanted to make a gem dependency hierarchy that looked something like omniauth

  1. I would have a base gem that would not be of much use on its own which I will call parent gems (like the actual omniauth gem, which is never used alone), and
  2. I would have a few gems that would somehow depend on my base gem which I will call child gem (collectively children gems) (like the strategies provided for omniauth, e.g. omniauth-github.)

The vision was that for future projects to only include a single child gem (like omniauth-github) and all features and definitions in the the parent gem be accessible to the rails project, as well (e.g. monkey patches, etc.)

Update With time, I realized devgem has become an incredibly clean way to develop gems even if they are standalone. In another case, I am developing an interactive console-based gem and using devgem for development makes development/debugging/testing so much easier.

The Silly Solution

The tiresome silly solution

The tiresome way would be to install my parent gem AND my child gem every time I make a change to them. This way I would simply use the child-gem from my Gemfile and simply require my parent-gem from my child-gem (makes sense)


# Gemfile
...

gem 'child-gem-1'

...


# /path/to/child/gem1/lib/gemfile.rb

require 'parentgem'
...

As as I said, this makes sense and should be the way to go, except bundling every time is not a very time-friendly way to go.

The dumb silly solution

The silly solution consisted of two parts. First, the gems in my project would require a path option to tell bundler where to load my gems from. The second, more serious problem, was that I had to explicitly add my parent gem to the Gemfile for my project to be able to use the functionality provided there.


# Gemfile
...

gem 'parent-gem', path: '/path/to/parent/gem'
gem 'child-gem-1', path: '/path/to/child/gem1'

...

This would save me the hassle of having to `rake install` my gem file after every step, and instead keep my gem loaded from my development directory.

The Proper Solution – Devgem

Proposed by Florian, the idea is simple: after installing your gem locally, symlink to it. Genius! So the typical usage scenario becomes:

  1. Create gem
  2. Bundle it and install it
  3. Remove the source found in $GEM_HOME/gems/child-gem-0.0.1
  4. Create a symlink to your development source instead

And here’s where devgem comes into play: it performs just that for you (needless to say, devgem is a few lines of code.) You can check the source, installation steps, and usage on Github.

Alternative solutions

Alternatively, you could run a gems server that would handle your updates locally, but that would require adding an extra layer above your project development process. Make your pick.

Happy development!

Is neo4j really as good as it says it is? Maybe not.

There’s a long introduction. For the interesting juice, click here

Whatever new technology comes along, it is accompanied with a “great deal.” I don’t just mean an above normal coverage, I mean a big promise that you’re getting a great deal by using/switching/adopting/etc.

In neo4j’s case, it was no different.

During our Information Management course, we had to dig into graph databases and neo4j was right there waiting for us. In the process (of digging), we were sucked right in. All information we read were pointing in just one direction — neo4j is awesome.

And maybe it was. But there was a problem — why wasn’t there “another side of the story” anywhere? All strong comparisons were posted to prove that neo4j was better in doing certain tasks than the most common RDBMS out there — MySQL.

The Midterm

After spending 6 weeks digging into neo4j and believing all the resources we read (and we read a lot! Online resources, literature, etc.) we showcased to our class that neo4j was amazing. However, our Professor was not convinced. Instead, she asked us: could there be a neglected side to all this?

We set journey to at least try and duplicate these data to see if they had any truth to them. We came across this blog post by Jörg Baach, with exactly the same goal as us — seeing if it was the entire truth. Baach had posted the code he used for his tests, which we based our work on. (His original work could be found here.)

The information that we wanted to duplicate and at least have a go at were retrieved from Baach’s post, and this paper as well.

Benchmarking

Our code is on github: https://github.com/kamasheto/neo4j-exps

The work we did consisted of four main tasks:
1. Installing the appropriate software (and tuning as necessary)
2. Generating the appropriate data
3. Importing the generated data in both database engines
4. Running and reporting the tests

Installing database engines

We installed neo4j as normal, and used the properties provided by Baach to further tune the setup. This set of properties added indexing features and increased the buffer size of neo4j. Refer to Baach’s blogpost for information on how to tune both MySQL and neo4j further.

The requirements are, however:
– Python (with MySQLdb module)
– Neo4j and MySQL (obviously)

Generating the appropriate data

To generate the data, we ran `gen.py`:

$ python gen.py 1000

This created a `data/pickle.1000` file that included 1000 friends information, with 50 outgoing relationships each. This was stored so we’d have identical information in both cases. (You can change the 1000 to any number you desire, but keep in mind the time factor.)

Importing the data

To import the data, we ran `import.py`

$ python import.py mysql 1000
$ python import.py neo4j 1000

Running and reporting

After warming the cache, we run the tests and reported times with the `tests.py` script

$ python tests.py mysql 1000 3
$ python tests.py neo4j 1000 3

The 3 in this case is the number of hops the script would need to test against. The query is ran 10 times with different starting points and average time is reported. The sample query is printed out in the first run for reference.

Results

The reported results were as follows — the paper reported:

        MySQL     neo4j
100
S0      19.56     8
S1      33        12.65
S2      111.334   19.57
500
S0      281.38    10
S1      333.96    17
S2      620.56    21

This set of results clearly showed neo4j outperforming MySQL in every criteria tested. Baach, however, reported the exact opposite:

        MySQL     neo4j
Baach
100k
S0      0.000     0.010
S1      0.001     0.018
S2      0.072     0.376
1M
S0      0.000     0.010
S1      0.002     0.017
S2      0.082     0.484

His results showed MySQL in fact outperformed neo4j (he even reports his python implementation was even faster than both!) We reported results that were quite different than both:

        MySQL     neo4j
Our Results
100k
S0      0.0050    0.0417
S1      0.0357    0.1503
S2      1.3740    1.1313
S3      64.547    ∞

Discussion

The following remarks could be made about the obtained results

  • H0: MySQL has great indexing capabilities, allowing it to outperform neo4j up to two levels of relation hops
  • H1: neo4j’s graph traversing powers come to use starting to query more than three levels of relationship hops
  • neo4j is extremely resource intensive. This might be the reason behind it not reporting results for four levels of relationship traversal. We speculate it would outperform MySQL if it did complete.
  • Needless to say, tuning plays a great role in the execution times. Further tuning would be needed to produce more robust results

Twitter Bootstrap Callouts

Callouts are used on the Bootstrap docs to highlight important contents (hence; callouts). Unfortunately, they are not shipped with base bootstrap.

To add (and use) Twitter bootstrap callouts, here is the additional markup you need

/*
 * Callouts
 *
 * Not quite alerts, but custom and helpful notes for folks reading the docs.
 * Requires a base and modifier class.
 */

/* Common styles for all types */
.bs-callout {
  margin: 20px 0;
  padding: 20px;
  border-left: 3px solid #eee;
}
.bs-callout h4 {
  margin-top: 0;
  margin-bottom: 5px;
}
.bs-callout p:last-child {
  margin-bottom: 0;
}
.bs-callout code {
  background-color: #fff;
  border-radius: 3px;
}

/* Variations */
.bs-callout-danger {
  background-color: #fdf7f7;
  border-color: #d9534f;
}
.bs-callout-danger h4 {
  color: #d9534f;
}
.bs-callout-warning {
  background-color: #fcf8f2;
  border-color: #f0ad4e;
}
.bs-callout-warning h4 {
  color: #f0ad4e;
}
.bs-callout-info {
  background-color: #f4f8fa;
  border-color: #5bc0de;
}
.bs-callout-info h4 {
  color: #5bc0de;
}

About Being #ForeverAlone

People are horrible.

They’re unpredictable, and unless you live in heaven, they’re usually full of shit. People are boring. Not just the boring kind, but the fun kind too. Everything eventually is boring. People have different schedules. And different tastes. And they object and have opinions. They argue and make it awkward. They don’t get all your jokes. And some even bother you. Granted, unintentionally if they’re nice. But they do.

In my conversations with fellow humans, I like to indulge in the look of shock when I mention I go to the movies alone. More times than I had been with people, as a matter of fact. This is just one of the things in life I enjoy doing alone. I enjoy that same look on movie clerks when I book seats for a movie and I unapologetically pick just one spot. Heck, I once really thought I needed to reward myself so I booked two movies back-to-back.

Continue reading

The Barrel With Few Strings

I once took part in a soft skills course that illustrated our brain keeps memories in a barrel kind of fashion. A lot of everything is stored in sequence, once you remember something or pull on any string, a whole bunch of things come floating up for easier access.

When I was 18 years old my friends and I had been hanging out at a regular internet cafe, play Medal Of Honour among other games (I always won), and just casually have a chat when we’re waiting for our next course or something.

The place clerk has got to have had a very rough love life because for as long as I could remember, he used to play the above song. Day and night. It was just always on.

And for the string/barrel theory mentioned above, whenever I remember anything from the old times, for whatever reason, this song comes right up and I play it in my head a million times.

Mahmoud Sakr, kamasheto, Mahboud, and everything in between

%d bloggers like this: