Configuration objects in Scallop

It turned out that spreading type annotations throughout all the code in all places where options are requested is not a good idea. And, of course, it is prone to errors - who likes those type annotations?

When using such option parsing libraries, users usually end up creating a big object with fields corresponding to each option:
object Conf {
  var apples:Int = 0
  var bananas:Int = 0
}
// ... parsing options ...
Conf.apples = parser.getOption("option name", ...)
Conf.bananas = parser.getOption("other option name", ...)
Sure, this works, and usually flexible enough - but why would anybody want to create such boilerplate? And by the way, nobody likes var's in their code :)
So in Scallop, I needed to cut down that unnecessary repetition. There are several ways:
  • Create a compiler plugin. The most powerful option. And the least enjoyable - even given the ease of installing a compiler plugin using sbt, I doubt that anybody will install compiler plugin just to parse the command-line options.
  • Use macroses. Great choice, but it seems that it would not work for a long time - SIP-16 didn't make it into 2.10 (only behind -Xexperimental option). That and the fact that people are often scared of macroses :)
  • Mix that conf object creation with option definition. I decided to go with this choice, at least while we do no have those macroses.
So the following is an example of the result:
object Conf extends ScallopConf(List("-a","3","-b","5","tree")) {
  val apples = opt[Int]("apples")
  val bananas = opt[Int]("bananas")
  verify
}
Conf.apples() // 3
Conf.bananas.get // Some(5)
The only problem here is that "val apples" contains not the value itself, but the ScallopOption[Int] instance. To extract the actual value, you will need to call either .apply() method (returns a value), or .get method (returns the value as an Option). 

ScallopOption is needed to wait until the rest of the option set is defined - since if you try to get the value on-the-spot, the parser will know nothing about the "bananas" option and the trailing argument - which can result in some bad things sometimes.

To protect you from this scenario, ScallopConf checks that .verify method was called before any options are extracted. So if you want to define some custom extractor for value, you'll need to either use a lazy val, or define your extractor after the .verify call:
object Conf extends ScallopConf(List("-a","3","-b","5","tree")) {
  val applesO = opt[Int]("apples")
  lazy val apples = apples.get.map(a => if (a > 3) "plenty" else "few").getOrElse("zero")
  val bananas = opt[Int]("bananas")
  val name = trailArg[String]()
  verify
  val bananasValue = bananas() + 3
}
Conf.apples // "few"
Conf.bananasValue // 8
This will work, and type-safety is still in place, but inventing another name for an extractor or suffixing the option value with something looks ugly. We surely can do better :)

In fact, ScallopOption is just a laziness wrapper around standard Option - that is, it batches all operations on it, and evaluates the value as the last resort measure. It has most of standard Option methods defined - like .map, .orElse, .collect, etc.

So that snippet can be written a bit cleaner (without lazy vals and redundant val names):
object Conf extends ScallopConf(List("-a","3","-b","5","tree")) {
  val apples = opt[Int]("apples").map(a => if (a > 3) "plenty" else "few").orElse(Some("zero"))
  val bananas = opt[Int]("bananas").map(3+)
  val name = trailArg[String]()
  verify
}
Conf.apples() // "few"
Conf.bananas() // 8
Almost perfect. It is so close to complete boilerplate-lessness, that the advantages of the macro in place of all this are quite questionable.

But there are still some advantages - notice that "apples" is appearing twice on that line: in a val name and in the option definition itself. And forcing the programmer to remember to extract the options in the right place isn't good either.

So, wait for the macro version :)

The code is located on github, the apidocs are there as well, comments, suggestions, and forks are all welcome!

Comments

  1. Related interest:

    https://github.com/scala/scala/tree/master/src/compiler/scala/tools/cmd

    ReplyDelete
  2. i have a small request, if that's ok,

    i just ran into scallop today for the first time, and i gotta say, i like it! good job!
    one thing that realy annoyed me though, was when i tried to make my own ValueConverter.
    maybe my scala skills aren't sharp enough, but no matter the reason, i was not been able to do what i wanted.
    which leads me to my humble request:

    my need was to have a bunch of boolean properties,
    much like the jvm arguments "-Dsome.property=true -Dother.property=false".

    so i wanted to use something like that:
    object Conf extends ScallopConf(args){
    val properties = props[Boolean]('D')
    }

    but there is no suitable implicit value to match a Boolean type.
    so i tried to create one explicitly, and wasn't very successfull.
    so if you could add a small implicit value,
    something that would look kinda like:

    implicit val booleanPropsConverter = propsConverter[Boolean](singleArgConverter[Boolean]{s =>
    s.toLowerCase match {
    case s if(List("true","t","y","yes").contains(s)) => true
    case s if(List("false","f","n","no").contains(s)) => false
    case _ => /*
    * not implemented. return None? (return value would be Option[Boolean]
    * instead of Boolean...
    * maybe have a default value? would it be user-definedable?
    */
    }
    }
    )

    it would be awesome!
    thanks again for a great CLI-tool!

    Gilad.

    ReplyDelete
    Replies
    1. Hello! Thanks for your comment.

      Actually, you almost created that converter - you needed only to step a bit forward. First of all, observe that props[T] returns (String => Option[T]) - so you can decide which value to use as default, when unboxing that option.

      Here's the small example of creating that converter:

      scala> implicit val booleanPropsConverter = propsConverter(singleArgConverter({
      case "true" => true
      case _ => false
      }))
      // now, some usage
      object Conf extends ScallopConf(Seq("-Dsome.property=true")) { val properties = props[Boolean]('D') }
      Conf.properties("some.property")
      res1: Option[Boolean] = Some(true)

      I'd like to avoid adding this converter to the library, mostly because of the hard choice of what to consider "truthy" values (in your example - true, t, y, yes) and what to consider "falsy" values.

      Actually, if I needed to do such conversion, I probably would do it the following way ("in-line"), which is quite short. Again, here I exploit the fact that we have (String => Option[T]) in our possession:

      object Conf extends ScallopConf(Seq("-Dsome.property=True")) {
      val properties = props[String]('D') andThen
      (_.map(_.toLowerCase)) andThen
      (s => s.map { case "true" => true case _ => false })
      }
      Conf.properties("some.property")
      res5: Option[Boolean] = Some(true)

      I hope this helped.

      Best wishes,
      Rogach

      Delete
    2. it helped a lot! thanks. code finally resulted with:
      https://gist.github.com/4378205

      well, regarding the implicit value... if you want to add something more generic,
      that lets the user choose the "falsy" and "truthy" values,
      you could have an implicit method instead of a value.

      i.e. a method that excepts (falsy: List[String], truthy: List[String])
      where's (falsy.isEmpty || truthy.isEmpty) //but not both
      for default values, or, let it fail with an exception like i did,
      in case both lists are'nt empty.

      thanks again!

      Delete
    3. Hello, I'm trying to use Scallop in a scala project for the first time and have some difficulties with it. Could you please help me?
      What I'm trying to achieve is such parameter set:
      app ... load user account info
      app -c -f -l ... create a new account
      app -u --disable
      app -u --enable
      and so on.

      I was not able to come up with a proper solution so I made some changes.

      My code:
      class Conf(arguments: Seq[String]) extends ScallopConf(arguments) {
      val mail = opt[String](required = true, descr = "User's e-mail address")
      val create = new Subcommand("-c") {
      val firstName = opt[String](required = true)
      val lastName = opt[String](required = true)
      }
      val read = opt[Boolean](descr = "Read user information")
      val update = new Subcommand("-u") {
      val disable = opt[Boolean](descr = "Disable user account")
      val enable = opt[Boolean](descr = "Enable user account")
      mutuallyExclusive(disable, enable)
      }
      addSubcommand(create)
      addSubcommand(update)
      verify()
      }

      I can check if a 'read' operation has been requested:
      if (conf.read()) {
      // do something
      }

      How can I check if a 'create' operation has been specified?
      I've used this:
      if (conf.create.firstName.isSupplied == true) {
      ...
      }
      but the code throws exception:
      "Exception in thread "main" org.rogach.scallop.exceptions.UnknownOption: Unknown option 'f'
      on this line:
      if (conf.update.enable()) {

      which doesn't make sense to me to be honest.
      Thanks. :-)

      Delete
    4. Hi! Can you please give the arguments that you use to get that exception?

      I got the same exception only when providing single "-f" option (nothing else), in which case it makes sense (no subcommand is provided, and there is no such option on root configuration.

      P.S. Can we please move this discussion to issue tracker on github? Comments screw code formatting here.

      Delete
    5. Hi! Thanks for a quick response. I'll create a new issue in the tracker.

      Delete
    6. This is a badly done blog post. Your second example makes no sense at all. What the hell is the arguments " List("-a","3","-b","5","tree") " supposed to do.

      How do I directly pass scala args to scallop conf, what happens.

      Learn how to present a library in a lucid way possible. You will have way more traction that way.

      Delete
  3. I take this opportunity to digress a little and point out that the wiki page for 'help' is very unhelpful to a new user. I tried it all sorts of ways but couldn't understand anything. The difficulties

    1. You say running a command produces output but please WRITE the command too CLEARLY.
    2. I tried creating the config in two ways and tested both separately but couldn't get any help to be printed.
    1. class Conf extends ScallopConf(Seq("--help")) {}
    2. class Conf(arguments: Seq[String]) extends ScallopConf(Seq("--help") { }
    After this I tried printing help by trying : Myapp.jar --help but it just threw up an error!

    Please provide better example to run help and use with the commands
    Thanks,
    Varun

    ReplyDelete
    Replies
    1. I hope "Basic usage" page from the wiki will help you to get started: https://github.com/scallop/scallop/wiki/Basic-usage

      Also it would be much easier to help you with this issue if the entire code that you used will be available - best place to post it will be in a GitHub issue here: https://github.com/scallop/scallop/issues

      Delete
  4. Hi rogash, why it is not possible to have other extra arguments without declaring them. Consider below code snippet

    class Arguments(arguments: Seq[String])
    extends ScallopConf(arguments) {
    val id: ScallopOption[String] = opt[String](required = false)
    verify()
    }
    object Main {
    def main(args: Array[String]): Unit = {
    new Arguments("--id", "123", "--other_option", "23")
    }

    ReplyDelete

Post a Comment

Popular posts from this blog

How to create your own simple 3D render engine in pure Java

Solving quadruple dependency injection problem in Angular