Depending on your project, you may reach a point where running your tests consumes an extraordinary amount of memory, or you're heading that way. If you're not sure, take a look at how much memory your test runs consume, you may be surprised. This usually kicks in when you're hitting around 1,000 tests and have a sizable number of controllers, services, and directives. Our test environment is comprised of Karma, Mocha, Sinon-Chai, Angular Mocks, and PhantomJS. This problem recently kicked in for us and were were locked out from passing builds on Travis for the day.

We learned that by the end of our tests, PhantomJS's memory footprint was exceeding 2GB. Ready to pin the problem on PhantomJS, we tried running our tests in FireFox and Chrome, as there had been some mentions of pre 2.0 PhantomJS leaking memory - angular/material#4734, ariya/phantomjs#12317, as a couple examples. The same results were apparent in all environments.

I was eager to blame this problem on a memory leak, either in my code or with Karma itself, but having to deal with this problem I had the chance to learn a little bit more about the internals of Angular and Angular Mocks.

Profiling

After restarting Travis five times, I began monitoring the memory usage after each unit. I omitted all my unit tests except for a set of units used profile and observe any patterns. The test is set up as the following:

• Load the angular module before each test
• Call inject() without any services (maybe those were the problem and not getting recycled)
• Run a ton of basic units, and throw in a timeout so there is time to evaluate each step

'use strict';
describe('memory test', function () {
  beforeEach(module('app'));
  beforeEach(inject());
  describe('testing', function () {
    it ('should grow in memory', function (done) {
      setTimeout(function () {
        for (var i = 1; i < 100; i++) {
          var result = Math.pow(i, 2);
          expect(result).to.be.eql(Math.pow(i, 2));
        }
        done();
      }, 500);
    });
    // repeat the above unit 500 times
  });
});

I watched every unit tick by with more and more memory being consumed even without my tests doing anything exceptional, and without executing any application code or injecting any services that may be leaking:

My next move was to remove beforeEach(module('name')) and see if this changes my memory footprint. It didn't really.

'use strict';
describe('memory test', function () {
  // beforeEach(module('app'));
  beforeEach(inject());
  describe('testing', function () {
    it ('should grow in memory', function (done) {
      setTimeout(function () {
        for (var i = 1; i < 100; i++) {
          var result = Math.pow(i, 2);
          expect(result).to.be.eql(Math.pow(i, 2));
        }
        done();
      }, 500);
    });
    // repeat 500 times
  });
});

As nothing changed, my last attempt was to remove the inject call.

'use strict';
describe('memory test', function () {
  // beforeEach(module('app'));
  // beforeEach(inject());
  describe('testing', function () {
    it ('should grow in memory', function (done) {
      setTimeout(function () {
        for (var i = 1; i < 100; i++) {
          var result = Math.pow(i, 2);
          expect(result).to.be.eql(Math.pow(i, 2));
        }
        done();
      }, 500);
    });
    // repeat 500 times
  });
});

And there we had it. My memory didn't move, and if it did by the last few tests, it quickly dropped back down to 119MB. It became clear that when we call inject, things were being created and they weren't being recycled. I dove into the source code for angular mocks, which defines window.inject, but it was nothing more than a wrapper for angular.injector and $injector.invoke, which produces the $injector service and resolves the dependencies.

It's basically an easy way to do:

var injector = angular.injector(['ng', 'app']]);
injector.invoke(function (ServiceA, ServiceB, ..) {
  // your injected services
});

It seemed as if the problem was out of our hands.

Solution

As we clearly don't want to muck around with the internals of angular and its most important service, $injector, we decided to do the easy thing. The "easy thing" is usually what an engineer may overlook.

We split up our unit tests into multiple runs. Instead of one giant run, we had a handful of runs:

//..
karma: {
  controllers: {
    configFile: 'karma.conf.js',
    options: {
      files: karmaFiles.concat(['angular/test/spec/controllers/*.js']),
      port: 8080,
      proxies: getKarmaProxies(8080)
    }
  },
  directives: {
    configFile: 'karma.conf.js',
    options: {
      files: karmaFiles.concat(['angular/test/spec/directives/*.js']),
      port: 8081,
      proxies: getKarmaProxies(8081)
    }
  },
  ...
}
//..
grunt.task('unittest', [
  'karma:controllers',
  'karma:directives',
  //..
]);

As we were no longer limited to a single browser instance that couldn't support all the tests, our units were able to execute in batches without exceeding any memory limits. The tests were broken down into groups that ran separately -- controllers, directives, services, etc. We updated our Gruntfile to add a handful of new rules to Karma, threw up a build on Travis, and watched it successfully build without failing.

Lessons & Moving Forward

If your tests are growing too large for your environments to handle, break them up into multiple runs.

Also consider this rule for your angular module itself. We currently have things living in one big house, when they should really be broken down into different modules ('app.services', 'app.controllers', etc). This provides for a much lighter footprint when module() and inject() get called with angular mocks. It will produce a much lighter object, rather than carrying the entire weight of the application. This will be one of the next chores we do for improving our test environment.

I hope this helps someone trying to narrow down their problem. If you've solved this problem another way, I'd love to hear what you did below in the comments.