Dynamic versioning according to namespace

[nirzamir]

Hi,

Consider a namespace-based versioning of the same controllers/models (like in the code examples here).
I was thinking of using convention-based versioning according to the namespace of each controller - go over all controllers, extract the version from their namespace, and version the controller with that version.
This way, when I'm working on a new version, I replicate the current version to a new folder, refactor the namespaces, and that's it.

Will that work? Did anyone try it (both code-wise and maintenance-wise)? Any #implementation tips are welcome... I'm using ASP.Net Web API.

Thank you!
Nir

[commonsensesoftware]

A few people have asked about it, but I've never seen anyone implement it - yet. Believe it or not, it's relatively simple to setup. The real challenge is coming up with a namespace naming convention that the compiler (say C#) will be happy with. I would suggest the following BNF format:

'v' : [<year> '-' <month> '-' <day>] : [<major[.minor]>] : [<status>]

This would preserve full fidelity with the API version conventions with albeit ugly namespaces. Here's a few examples:

• namespace Contoso.Api.v1.Controllers { } -> "1.0"
• namespace Contoso.Api.v1_1.Controllers { } -> "1.1"
• namespace Contoso.Api.v0_9_Beta.Controllers { } -> "0.9-Beta"
• namespace Contoso.Api.v2018_04.Controllers { } -> "2018-04-01"
• namespace Contoso.Api.v2018_04_01_Beta.Controllers { } -> "2018-04-01-Beta"
• namespace Contoso.Api.v2018_04_01_1_0_Beta.Controllers { } -> "2018-04-01-1.0-Beta"

To parse this, I would use an extension method like this (mind the nasty regular expression):

static class TypeExtensions
{
    internal static ApiVersion GetApiVersion( this Type controllerType )
    {
        // 'v' : year-month-day : major[.minor] : status
        // ex: v2018_04_01_1_1_Beta
        const string Pattern =
                @"[vV](\d{4})?_?(\d{2})?_?(\d{2})?_?(\d+)?_?(\d*)_?([a-zA-Z][a-zA-Z0-9]*)?";

        var match = Regex.Match( controllerType.Namespace, Pattern, RegexOptions.Singleline );
        var value = new StringBuilder();

        if ( match.Groups[1].Success && match.Groups[2].Success && match.Groups[3].Success )
        {
            value.Append( match.Groups[1].Value );
            value.Append( '-' );
            value.Append( match.Groups[2].Value );
            value.Append( '-' );
            value.Append( match.Groups[3].Value );
        }

        if ( match.Groups[4].Success )
        {
            if ( value.Length > 0 )
            {
                value.Append( '-' );
            }

            value.Append( match.Groups[4].Value );

            if ( match.Groups[5].Success )
            {
                value.Append( '.' );

                if ( match.Groups[5].Length > 0 )
                {
                    value.Append( match.Groups[5].Value );
                }
                else
                {
                    value.Append( '0' );
                }
            }
        }

        if ( match.Groups[6].Success && value.Length > 0 )
        {
            value.Append( '-' );
            value.Append( match.Groups[6].Value );
        }

        return ApiVersion.Parse( value.ToString() );
    }
}

Now that you have a way to get an API version by convention from the namespace, you can use the API versioning conventions to configure the application. This is suprisingly easy to do using:

configuration.AddApiVersioning(
    options =>
    {
        options.ReportApiVersions = true;

        var services = configuration.Services;
        var assembliesResolver = services.GetAssembliesResolver();
        var controllersResolver = services.GetHttpControllerTypeResolver();
        var controllers = controllersResolver.GetControllerTypes( assembliesResolver );
        var conventions = options.Conventions;

        foreach ( var controller in controllers )
        {
            conventions.Controller( controller ).HasApiVersion( controller.GetApiVersion() );
        }
    } );

I was able to apply this change and make it work with the existing By Namespace Example with minimal effort.

In general, the right thing just happens. Any explicit attribute definitions will still be honored if they are present. This will result in a union for the conventions applied to a controller. In terms of maintenance, there's some addition things to consider:

1. Refactoring/renaming a namespace could have dire consequences. Do so with caution.
2. Version interleaving is still possible using explicit attributes, but it's messy. I would avoid this.
3. Deleting a controller or namespace deletes the service, which is honestly probably what you want.
4. If you support side-by-side (SxS) pre-release versions in independent namespaces, it will definitely work, but it could be painful to maintain. This might be the one and only case where you change a namespace. In order words, you work with a namespace suffix like "_Beta", etc until you are ready to transition to production-ready. How you do this is up to you. You might want SxS for early adopters or you just might cut over to the release version.

I've often thought this would be a pretty useful feature. Having proven to myself that it could work and is generally what large, mature service codebases will want, I think I might queue this up as a new feature in the next major release. I think you'd simply opt into the feature with a new option. Something like options.DeriveApiVersionFromType = true. Until then, you have all the extensibilty points to make this work now. :)

[nirzamir]

Thank you very much for the detailed answer!
I will try it out soon.

Regarding namespace renaming and side-by-side - I'm not sure I completely understood what you mean, but when using the methodology I had in mind, when I replicate the current version to a new one I'm still not sure what the released version would eventually be, since I might not know for sure if the new version will have breaking changes etc.
That's why I need the ability to rename the folder and namespace of the new version development phase - I think there are tools that can make this task quite easy.

I think what you meant with SxS is when I want to release a beta version for early adopters, in which indeed I'd need a "_Beta" suffix or alike.

Did I get it right?

[commonsensesoftware]

Correct. If you transition from say "2.0-Beta" to "2.0" and there aren't any implementation changes, you'll undoubted be tempted to just update the namespace. Unfortunately, that would break all the "Beta" clients because the version will no longer exist. That's not a huge deal if that's your stated policy. Alternatively, you can also add [ApiVersion( "2.0-Beta", Deprecated = true )] to all the controllers at the same time you rename the namespace. "2.0" and "2.0-Beta" would be aggregated. This is counter to your goal of zero touch, but it's an option.

This surfaces an additional point on maintenance. At some point, some API versions will likely become deprecated. There's no clear way to do that by convention. One possibility would be to have a "Deprecated" namespace. For example, Contoso.Api.Deprecated.v0_9. The ApiVersion type doesn't know whether it's deprecated (by design). You'd need an additional extension method to extract this information. You can you can have something like:

foreach ( var controller in controllers )
{
    if ( controller.IsDeprecated() )
    {
        conventions.Controller( controller ).HasDeprecatedApiVersion( controller.GetApiVersion() );
    }
    else
    {
        conventions.Controller( controller ).HasApiVersion( controller.GetApiVersion() );
    }
}

Hopefully that gives you some more ideas. I've queued up #278 as a proposed feature for 3.0. Any additional thoughts, experiences, or limitations you run into with this approach would be insightful.

[nirzamir]

Hi,
while testing it I found out that the regex doesn't match some namespaces properly.
Take this one for example:
"MyRestaurant.Vegetarian.Food.v1_1.Controllers"
The 'V' of 'Vegetarian' messes this up... the reason it worked for you in the example code is that it
doesn't have another 'V' in the namespace... In addition, I'd add '.' in the beginning or the regex.

In my case I can change your code to go over ALL matches until the extracted version is not empty, but I thought I'd let you know for the feature you're planning.

[nirzamir]

@commonsensesoftware just making sure you saw my comment :)

[commonsensesoftware]

I did. Thanks for the heads up. I'll be sure to add that scenario as a test case. ;) Thanks!

[nirzamir]

@commonsensesoftware Great! Happy to help.

One more thing I wanted to consult about (please let me know if it should be in a new issue):
Assume I implement the above versioning by convention, and want to create minor versions when there are no breaking changes. That means, that for every API version I start working on:

1. Replicating ALL the version folders containing controllers and models/mappers
2. Rename the folders and refactor the namespaces (and their references) in the files with the new version.
3. The above also applies to any unit/system test projects of the API or replicated classes

Although it makes sense to me, and although the duplicated code is in classes that are supposed to be lean with no business logic, I wanted to make sure I'm not missing something here. Since our current approach is creating minor versions for non-breaking changes (to help clients keep track of changes they receive, and truly be non-breaking), adding a new version will happen quite a lot.
Also, and I know it's a little out-of-scope, I'd be happy for any advise on how to automate the above steps or anything to make the process of adding a new version easier. I believe it can be useful to anyone using this approach.

Your thoughts are most welcome!
Thanks

[commonsensesoftware]

In my opinion, there is always a misnomer in services about what a breaking change is. Any change in contract can and should be considered breaking. Many service authors assume that additive changes are non-breaking, but that simply is not the case. You cannot guarantee the client follows the Tolerant Reader pattern. You usually also cannot make assumptions about what client technologies will be used. While it's true that many mature client technology stacks will support tolerant readers, it may not be true for all. This is one of the reasons why all API versioning must be explicit. A minor version technically doesn't imply anything officially, but based on your conventions or arguably commonsense, the changes are small. Small changes can still be breaking.

Using pure folder-based conventions, it doesn't take long before this approach can get nasty with minor versions. There's no one magic answer, but there's a couple of reasonable approaches:

1. Continue the pattern; you just end up with a bunch of folders.
2. Interleave minor versions. This may be acceptable for small, additive changes that occur infrequently and don't warrant a whole new folder.
3. Establish and communicate a clear versioning policy; for example, we support N-2 versions. This will help mitigate the need to support a ton of different versions.

Having a clear versioning policy, especially if it supports minor versions (date-based doesn't), is likely to help you influence your decision. You release/versioning cadence will also influence the decision. If you only occasionally deal with minor version changes, then interleaving might be the simplest way to deal with those one-off cases.

I hope that helps give you some ideas.

[nirzamir]

Thanks a lot for your answer.
We have a release cadence of every 2 months at the moment (might be 1 month in the future).
Regarding interleaving, I'm just afraid that it will be messy and difficult to compare two versions, and interfere the automation of initializing a new version/folder. We also have quite a few developers working on a version (we have quite a few endpoints in the API).
I will definitely push to establish a policy so we don't end up with tons of versions, but it seems like this pattern is the clearest one. If you think otherwise, I'd be happy to hear.

Thanks again!
Nir

[commonsensesoftware]

Sounds reasonable. In service teams that I've worked in the past, we moved to the date-based model with this type of cadence. It just became a lot easier to rationalize about. In the early iterations, numbers are useful, but after a year or so, their meaning became nearly useless. In terms of matching releases to API versions, it was much easier to talk in terms of dates. Other large teams like Azure do the same thing.

You'll have to decide if it's right for you and your teams, but this might be a better strategy for you. With a N-2 policy, if you're constantly releasing, the oldest thing you'll support is 6 months old - until you stabilize. After you stabilize, you'll know how old something is purely by its API version.

I hope that helps.