Not only parse, also generate

[mixtur]

So basically what your library does is extracting some data structure from url based on certain pattern.
Can you please implement the opposite thing. I mean some method to which you feed data structure and get url string in return.

Something like

var pattern = new UrlPattern('/posts/:start/:count/:whatever');
assert(pattern.generate({
  start: 1,
  count: 100,
  whatever: 'whatever'
}) === '/posts/1/100/whatever');

Such feature would allow (me) to write routers together with History API manipulation in browser quite easily.

[snd]

hey michael,

So basically what your library does is extracting some data structure from url based on certain pattern.

exactly!

url-pattern checks whether a string matches a pattern and if it matches it extracts the right values for the named segments in the pattern from the string and returns them as an object.
it's not that easy which is why i wrote a library for it.

the opposite is just string replacement.
i wrote a simple (untested) function that should do what you want:

> var generate = function(string, object) {
  for (var key in object) {
    string.replace(':' + key, object[key]);
  }
  return string;
};

> generate('/posts/:start/:count/:whatever', {
  start: 1,
  count: 100,
  whatever: 'whatever'
});
'/posts/1/100/whatever'

i am hesitant to put this kind of functionality into the library.
i want url-pattern to stay focused on parsing and matching.
still thinking about it though...

by the way:
we usually solve this problem (the problem of url generation) by having a function for each url that
takes the url params as arguments and returns an url string.
we then use that function to generate both the patterns and the urls:

> var postsUrl = function(start, count, whatever) {
  return '/posts/' + start + '/' + count + '/' + whatever;
};

// let's make a pattern
> postsUrl(':start', ':count', ':whatever');
'/posts/:start/:count/:whatever'

// let's make an url
> postsUrl(10, 100, 'whatever');
'/posts/1/100/whatever'

we use a couple of helpers to keep this very DRY by reusing parts of urls. it works quite well.

hope this helps!

best,
max

[tunnckoCore]

@mixtur you can use https://github.com/jonschlinkert/rte for that thing.

[mixtur]

@tunnckoCore yes, for simple examples like the one I stated above rte will work just fine, but when it comes to more complicated cases like new UrlPattern('/v:major(.:minor)/') it won't work as expected.

[tunnckoCore]

@mixtur hmm, yea. Can you share me more fixtures, cuz I have something in mind? You can message me at tunnckoCore/messages, if not here.

[snd]

thought about this some more.
i think i'll add generate !
it's a better solution to the one we are using atm (that i described above).

some thoughts on implementing this:

generate is straightforward to implement for patterns without optional segments and without wildcards.
we could restrict generate to such patterns and throw an exception otherwise.
but then your second example wouldn't work.

we could handle optional segments by omitting (replacing by the empty string) all static optional parts (like a(bbb)c) and by replacing optional parts containing named segments (like a(b:named)c) only if those properties are set on the object passed to generate:

> var pattern = new UrlPattern('/v:major(.:minor)/');

> pattern.generate({major: 1});
'/v1/'

> pattern.generate({major: 1, minor: 2});
'/v1.2/'

> pattern.generate({});
// throws

generate should throw if a parameter that is not in an optional segment is missing.

if a segment name occurs more than once generate expects an array for that parameter with the correct number of elements and throws otherwise:

> var pattern = new UrlPattern('/a/:id/b/:id');

> pattern.generate({id: [1, 2]});
'/a/1/b/2'

> pattern.generate({id: 1})
// throws

wildcards in patterns would be omitted (replaced by the empty string) by generate unless _ is set on the object passed to generate in which case they are replaced by that:

> var pattern = new UrlPattern('/aaa/*');

> pattern.generate({});
'/aaa/'

> pattern.generate({_: 'bbb'});
'/aaa/bbb'

generate should (if possible) preserve the following invariants:

1. for any pattern and any string that matches pattern:
   string === pattern.generate(pattern.parse(string))

2. for any pattern and any object that is valid input to pattern.generate:
   object === pattern.parse(pattern.generate(object))

what do you think ?

[tunnckoCore]

erm.. Im confused, lol. maybe I should sleep a bit. 😴

btw, maybe .stringify make more sense for method name

[snd]

yes, .stringify is a much better name.

[mixtur]

@tunnckoCore for more fixtures you can use README.md in url-pattern
generate - is a little more computer-science-y for my tastes.
stringify/parse - is what you have in JSON so... I don't know. I like both options actually.

@snd about invariants: yes that feels natural
I would add to it a little bit though

1.1) for any pattern and any string that matches pattern:
string === pattern.generate(pattern.parse(string))
1.2) for any pattern and any string that doesn't matches pattern:
pattern.parse(string) - should throw an error

2.1) for any pattern and any object that is valid input to pattern.generate:
object === pattern.parse(pattern.generate(object))
2.2) for any pattern and any object that is invalid input to pattern.generate:
pattern.generate(object) - should throw an error

Also that implies exposing some way to check if string matches pattern, and maybe if object is valid for pattern.

[snd]

in my opinion stringify is a better choice because its more intuitive and less surprising:
JSON.stringify and pattern.stringify are very similar: both take an object and return a string.

the way to check if string matches pattern is simply pattern.match(string) or null !== pattern.match(string) (see the readme).

[snd]

this is not trivial.
i won't have time to work on it next week.
my subconscious will be working though ;-)
i'm sure once i have time i'll know quite well whether and how to implement this.

some current thoughts as a reminder to myself:

situation: currently the parser/compiler is a state machine that consumes the pattern while immediately building the regex. this is fairly easy and fast.

problem: to implement .stringify we need to extract some information from the pattern eg. we need to parse it again, since the regex is not useful here. the current parser/compiler discards information that is useful for .stringify.

solution:
replace the current parser/compiler:
tokenize the pattern.
then transform the tokens into an AST.
build the regex from the AST.
store the AST in the pattern object.

the implementation of .match won't change.
.stringify is straightforward to implement if we have an AST of the pattern!

this is a simpler and more elegant implementation overall.

[mixtur]

Maybe you shouldn't actually do it all by yourself. There are a few parser generators around. For example:
http://pegjs.org/
http://zaach.github.io/jison/
That is probably overkill, but as you said this is not trivial. Complicated tools for a complicated problem.

Actually whole AST approach opens up ability to process more complex patterns, something like

assert(isEqual(
    new UrlPattern('/((\w+:type)-(\d+:id)+:product)').parse('/cellphone-123-234'),
    {
      product: {
        type: 'cellphone'
        id: [123, 234]
      }
    }
))

...though that is really seems like overlkill for library called url-pattern

[snd]

thank you for the suggestions.
i definitely think using a parser generator here is overkill.
it's not trivial but i wouldn’t say its a complicated problem either.
i have a pretty good idea on how to implement this from scratch with simple and reasonably short code.

since url-pattern is used in several client-side-routers i would also like to to keep dependencies to a minimum.

as for more complex patterns:
your example is interesting.
parsing such complicated patterns is outside my goals of url-pattern.
there is a lot of features i could add but i don’t because they would introduce complexity in implementation, documentation, tests and maintenance.
i don't want url-pattern to end up reimplementing regexes ;-)

i think it’s best to parse such patterns with regexes. url-pattern plays nice with them:
https://github.com/snd/url-pattern#make-pattern-from-regex

however, i’m always open to change url-pattern if a much better idea for a simple pattern syntax comes along :)

[maddie927]

If this is something you add, I would suggest putting it in a separate file to explicitly require, or another module like url-generators or url-pattern-generators, as part of the reason for using a small library like this is keeping bundle size down and many users won't need that feature. : ]

[snd]

the new parser (which produces an AST rather than a regex string directly) is almost finished.
it was really straightforward to implement from scratch by using parser combinators.
the AST is then fed into another simple function to produce the regex string that is used by .match as before.
i'm very happy with the direction this is going: the code is becoming simpler, less side-effecty, more functional and more flexible.

this is taking its time because i'm busy with other projects.
also because there are some design/API decisions i've yet to make that appeared while implementing it.
i'll write more about that soon!

@spicydonuts i'll keep that in mind. don't worry though. i don't think the filesize will change significantly.
the actual .stringify function will be few LOC.
after compression the difference should be negligible!

[mixtur]

is there any progress?

[snd]

here's a quick update:

while implementing this feature i rewrote the parser to use parser combinators. this opened up some exiting possibilities. i discovered some ways to significantly improve this library:

• having .match and .stringify roundtrip perfectly for all cases (x = match(stringify(x)) and x = stringify(match(x)))
• making * and : just two instances of a more general concept
• making it possible for users to add new instances of that concept to add syntax for numbers, uuids, ...
• making it possible for users to completely modify the syntax (*, :, ...). for example to easily change the syntax for named segments to <name>
• other things i’m not recalling right now...

i went bit too far and implemented most of it.
you can see some of the results here:
https://github.com/snd/url-pattern/tree/parser-combinators

i stopped because there were some unresolved questions/problems.
also this breaks backwards compatibility in a major way and i don’t want to do that right now.

i plan to release a new version within the next three weeks (i’m on vacation) with the following features:

• Not only parse, also generate #11 (this feature)
• regex named segments #14
• matching ~ #15
• new combinator based parser taken partially from https://github.com/snd/url-pattern/tree/parser-combinators
• backwards compatible except for rare cases of relying on old behaviour of matching ~ #15

i’ll continue to explore the ideas above after that.

[snd]

this is now published in 0.10.0.
click here for the relevant readme section.
there are some breaking changes. click to see the changelog entry.

[maddie927]

This looks awesome! Thanks!