I noticed when I was filtering for URLs ending in ".doc" that I was getting a lot of non-doc formats (HTML and text formats). I think it's less likely there will be such incorrect file extensions for the other binary formats we're targeting, but if we want a generic algorithm for determining the extension, I'd cover the .doc case by switching steps 1 and 2. I'd also use the plural method getExtensions and if the extension returned by FilenameUtils is contained in this list, I'd select that one.

We're already getting the MimeType with Tika, which is the only expensive operation in this process.

My method:

  def getExt(mimeType: String, url: String): String = {
    val tikaExtensions = DetectMimeTypeTika.getExtensions(mimetype)
    var ext = "unknown"
    // Tika method
    if (tikaExtensions.size == 1) {
      ext = tikaExtensions(0).substring(1)
    } else {
      // FilenameUtils method
      val fnuExt = FilenameUtils.getExtension(url)
      if (fnuExt != null && !fnuExt.isEmpty) {
        // Reconcile Tika list and FilenameUtils extension
        if (tikaExtensions.size > 1) {
          if (tikaExtensions.contains(fnuExt)) {
            ext = fnuExt
          } else {
            ext = tikaExtensions(0).substring(1)
          }
        } else { // tikaExtensions.size == 0 && fnuExt exists
          ext = fnuExt
        }
      } // else => unknown
    }
    ext
  }