Throttle

Last week, whilst working at The App Business, I came across quite a common problem. I was implementing an autocomplete search field where I wanted to perform a request to our backend API as a user updates the input field. When a user types fast, this sends a lot of requests which is quite costly, but also updating the results that quickly constantly updates the user with results they don't care about/are no longer relevant. This is jarring UX and causes your app to look cheap and designers to start throwing things at you.

My usual implementation

Usually when implementing this kind of feature I consider two options. The first option is to use a NSTimer and some kind of bool to throttle the calls to a function. The second option is to import a reactive library like RXSwift and use the throttle function you get out of the box.

Using something like RXSwift makes this task super easy but its adding another third party dependency to my code which isn't really needed, and personally I tend to find that when I add a reactive code to one section of my code it winds up everywhere. In this case its like killing an ant with a hand grenade. I usually end up creating some kind of implementation using NSTimer.

Not this time

This time however I decided to try and implement something cleaner. I didn't want any messy logic tracking state in a portion of my code that I have been trying to keep stateless. Lately I have been really interested in the AdvancedNSOperations talk given at WWDC last year. This talk and the awesome sample project released with it, kind of opened my eyes to the power of NSOperations. Until now I had tried to avoid them and in doing so missed out on some awesome features.

With all this in mind, I sought to solve my problem using operations. The approach was pretty simple, as always I started with the kind of type signature I wanted.

(NSOperationQueue, NSTimeInterval, () -> ()) -> Void

To create the behaviour I was seeking the approach was pretty simple. Create a NSBlockOperation with the passed in block and create a DelayOperation which takes the amount of time passed into the function to complete. Then by making the delay operation a dependency of the block operation I ensure the throttled behaviour.

struct Throttle {
  static func onQueue(queue: NSOperationQueue, by timeInterval: NSTimeInterval, function: () -> ()) {
    queue.cancelAllOperations()

    let delayOperation = DelayOperation(timeInterval: timeInterval)
    let throttledOperation = NSBlockOperation() {
      function()
    }
    throttledOperation.addDependency(delayOperation)
    queue.addOperations([delayOperation, throttledOperation], waitUntilFinished: false)
  }
}

The DelayOperation is a NSOperation Subclass that uses Grand Central Dispatch to complete after the appropriate amount of time. The reason it doesn't use the sleep() function is this is quite inefficient and it blocks the thread it was called on.

class DelayOperation: NSOperation {
  
  private let timeInterval: NSTimeInterval
  
  override var asynchronous: Bool {
    get{
      return true
    }
  }

  private var _executing: Bool = false
  override var executing:Bool {
    get { return _executing }
    set {
      willChangeValueForKey("isExecuting")
      _executing = newValue
      didChangeValueForKey("isExecuting")
      if _cancelled == true {
        self.finished = true
      }
    }
  }
  private var _finished: Bool = false
  override var finished:Bool {
    get { return _finished }
    set {
      willChangeValueForKey("isFinished")
      _finished = newValue
      didChangeValueForKey("isFinished")
    }
  }
  
  private var _cancelled: Bool = false
  override var cancelled: Bool {
    get { return _cancelled }
    set {
      willChangeValueForKey("isCancelled")
      _cancelled = newValue
      didChangeValueForKey("isCancelled")
    }
  }
  
  init(timeInterval: NSTimeInterval) {
    self.timeInterval = timeInterval
  }
  
  override func start() {
    super.start()
    self.executing = true
  }
  
  override func main() {
    if cancelled {
      executing = false
      finished = true
      return
    }
    let when = dispatch_time(DISPATCH_TIME_NOW, Int64(timeInterval * Double(NSEC_PER_SEC)))
    dispatch_after(when, dispatch_get_global_queue(QOS_CLASS_DEFAULT, 0)) {
      self.finished = true
      self.executing = false
    }
  }
}

This is a fairly standard implementation of an operation in Swift so I won't go into it in detail. With my implementation complete I started to have a play around with it and it seemed to work quite well, until I wrote my suite of unit tests...

Cancelling Async Operations

The times I was getting on my unit tests when I was calling the function multiple times seemed to not be right. The reason for this is the queue not being properly cleared out when I cancel one of my delay operations. With operations, calling cancel only sets a flag, it's up to the implementation to handle that change, and in this case it wasn't. When the max concurrent operations of the queue was set to one, we would still have to wait for the cancelled operation to finish if it had already started, slowing down our queue and given us incorrect behaviour.

To fix this I added the following override to my DelayOperation class,

override func cancel() {
    super.cancel()
    cancelled = true
    if executing {
      executing = false
      finished = true
    }
  }

This way if the operation is already executing when we call cancel, the operation will finish immediately.

Usage

Now my tests pass and my queue is clearing out properly, using Throttle feels like a nice elegant solution to the problem mentioned at the beginning of the post.

Throttle.onQueue(queue, by: autocompleteThrottleInterval) { [weak self] in
      self?.performAutocompleteSearch(searchString)
    }

Possible improvements

Currently throttle waits for the interval time before executing the function even if the queue is currently empty, we could modify the function to check if the queue is empty and if so execute the block straight away before adding a delay operation to the queue.

If you have any thoughts or comments please feel free to reach out to me on twitter.