Allow tokenizers and filters to be added as plugins

[clintongormley]

I'm wanting to add a tokenizer for email addresses, eg http://stackoverflow.com/questions/19014/using-lucene-to-search-for-email-addresses

It'd be great if we could add our own tokenizers/filters as plugins

ta

clint

[clintongormley]

That said - this is not a blocker for me. I've worked around it

[kimchy]

You can do that today (though in a clunky manner). I will add an API for that.

[kimchy]

The ICU plugin now uses this, so this can be done, though Java knowledge is required...