dotnet-host 2.1.0-preview2-26406-04-1 is not backwards compatible with 2.0.x applications: Failed to resolve library symbol hostfxr_main_startupinfo

[natemcmaster]

I'm seeing multiple reports that 2.0.x runtimes no longer launch due to failures in loading libhostfxr.so

travis-ci/travis-ci#9467
https://twitter.com/bradwilson/status/983782235777024000

Repro
On a clean machine, configure feeds as described on https://www.microsoft.com/net/download/linux-package-manager/ubuntu16-04/sdk-current
sudo apt-get install dotnet-sdk-2.1.104
dotnet --info

Expected
Stable releases of .NET Core should still work and only install stable versions of the dotnet-host package.

Actual
The installation brings down the prerelease dotnet-host package, version 2.1.0-preview2-26406-04-1, which issues this warning before exiting with and error code.

Failed to resolve library symbol hostfxr_main_startupinfo, error: /usr/share/dotnet/host/fxr/2.0.0/libhostfxr.so: undefined symbol: hostfxr_main_startupinfo

Workaround
Downgrade the dotnet-host package to an older version

sudo apt install dotnet-host=2.0.6-1

@steveharter @Petermarcu @leecow - preview2 impacting

cc @bradwilson @akoeplinger

[pakrym]

Is caused by this check. On windows returning null is allowed and it's checked in the condition here but on Linux PAL would log an error.

[bording]

Installing dotnet-sdk-2.1.104 also ends up installing the dotnet-runtime-deps-2.1.0-preview2-26406-04 package.

I don't think you should be publishing preview packages on the same feed as the release packages.

[steveharter]

So there are two issues here:

1. We are installing a preview version of the host when we shouldn't
2. There is a log message this is causing downstream issues. However, it is new to me that writing to stderr will cause csc.exe to fail since the actual process return value here should be success. Plus we write to stderr for informational cases when COREHOST_TRACE=1.

[steveharter]

Installing dotnet-sdk-2.1.104 also ends up installing the dotnet-runtime-deps-2.1.0-preview2-26406-04 package.

@livarcocc @nguerrera can you comment on this please

[nguerrera]

^ @wli3 Is there something wrong with the deb packages that is causing that? (cc @leecow)

[pakrym]

However, it is new to me that writing to stderr will cause csc.exe to fail since the actual process return value here should be success

A lot of applications treat child process printing to stderr as a failure (CI agents, MSBuild etc.) even when exit code is 0.

[leecow]

So there is 1 bug in the host package which Rakesh has fixed. This had the runtime-deps dependency on dotnet-host rather than dotnet-runtime. Because dotnet-host is the only package (at the moment) that is seen as upgradable by the package managers, the latest is always brought in.

A possible workaround for Ubuntu/Debian folks that have been hit by this is the following:

apt-cache madison dotnet-host : this will return a list of all versions available on the feed
apt-get install dotnet-host=[version you want from the list returned by madison]

[Petermarcu]

I think there is a larger problem here though. The host singleton needs to always work with all previous versions. It can't require an updated host policy or runtime.

[bording]

I also want to point out that @natemcmaster mentioned on twitter that the preview host package has been pulled from the feed, but that's left the 2.1.104 SDK installation broken because it complains about missing dependencies. The 2.0.6 host package has to be manually installed first before you can install the SDK right now.

[Petermarcu]

All of the preview packages got pulled but the dependency of the 2.1.104 SDK should still be there.

[bording]

@Petermarcu after using apt remove to remove all of the 2.1.104/2.0.6 packages and doing an apt update, when I run apt install dotnet-sdk-2.1.104, I get the following error:

Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 dotnet-sdk-2.1.104 : Depends: dotnet-runtime-2.0.6 but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

When I run apt-cache madison dotnet-host, I still see a preview package (though not the preview2 package):

dotnet-host | 2.1.0-preview1-26216-03-1 | https://packages.microsoft.com/repos/microsoft-ubuntu-xenial-prod xenial/main amd64 Packages
dotnet-host |    2.0.6-1 | https://packages.microsoft.com/repos/microsoft-ubuntu-xenial-prod xenial/main amd64 Packages
dotnet-host |    2.0.5-1 | https://packages.microsoft.com/repos/microsoft-ubuntu-xenial-prod xenial/main amd64 Packages
dotnet-host |    2.0.4-1 | https://packages.microsoft.com/repos/microsoft-ubuntu-xenial-prod xenial/main amd64 Packages
dotnet-host |    2.0.3-1 | https://packages.microsoft.com/repos/microsoft-ubuntu-xenial-prod xenial/main amd64 Packages
dotnet-host |    2.0.0-1 | https://packages.microsoft.com/repos/microsoft-ubuntu-xenial-prod xenial/main amd64 Packages
dotnet-host |    1.1.0-1 | https://packages.microsoft.com/repos/microsoft-ubuntu-xenial-prod xenial/main amd64 Packages

[leecow]

Just noticed that - working on getting the Preview 1 package removed.

[leecow]

Alrighty - run apt-get update then apt-cache madison and you should see no 2.1.0 Preview packages.

[bording]

@leecow Removing the preview1 package has gotten everything working correctly again.

[luto]

FYI: On a fresh CentOS 7 box installing dotnet-sdk-2.1.4 (or 2.1.3 or even 2.0.3) from the RHEL repo also pulls in dotnet-runtime-deps-2.1.0-preview2-26406-04. Trying to uninstall it afterwards removes all of dotnet again. I am seeing the same hostfxr_main_startupinfo errors as mentioned in the initial report.

Please just mention me if you need any additional information.

[steveharter]

The host singleton needs to always work with all previous versions. It can't require an updated host policy or runtime.

The host singleton (dotnet.exe) does work with all previous versions of hostfxr. The logging to stderr caused tooling issues here, and the PR above removes that logging.

The only thing that doesn't work is apphost (<mayapp>.exe) when it is from the 2.1 SDK and trying to use new features (relative path to framework) but only finds an old hostfxr then an error is raised.

[Petermarcu]

How is it that the 2.1 Preview host package being installed with 2.0 is failing then? I didn't see the almost scenario in this bug, just the dotnet.exe singletons.

[Petermarcu]

I talked to @steveharter and understand now that the host is tolerant, and doing the right thing functionally. The problem is only that it writes to stderr and some of our tools treat that as an error overall. We're going to explore options for getting an updated host package available.

[CRCinAU]

This seems to be broken still on the RPM repos. As a hack and workaround, I changed the dotnetdev.repo as follows:

[packages-microsoft-com-prod]
name=packages-microsoft-com-prod
baseurl= https://packages.microsoft.com/yumrepos/microsoft-rhel7.3-prod
enabled=1
gpgcheck=1
gpgkey=https://packages.microsoft.com/keys/microsoft.asc
exclude=*preview*

This allows us to rip & install any dotnet packages and not get the preview packages... Now to start doing this across our environments...

[npakudin]

CRCianAU's workaround helped for a clean CentOS 7

[LindaLawton]

Links to Failed to resolve library symbol hostfxr_main_startupinfo on Amazon Linux 2 AMI

[rudiv]

To fix an existing CentOS 7 installation that broke as a result of this bug, we removed the following:
yum remove dotnet-dotnet-runtime-deps-2.1.0-preview2-26406-04 dotnet-host dotnet-sdk-2.1.104

Then editing /etc/yum.repos.d/dotnetdev.repo and adding in the exclude line as @CRCinAU suggested and reinstalling dotnet-sdk-2.1.104 fixes it.

[CRCinAU]

A script friendly way to do this on machines you haven't already fixed up as per my suggestion:

echo 'exclude=*preview*' >> /etc/yum.repos.d/dotnetdev.repo
yum -y erase dotnet*
yum -y install dotnet-sdk-whatever

[ghost]

I confirm the suggestion made by @leecow regarding the setup in Ubuntu machines works by picking first the dotnet-host version by hand. Using Ubuntu 17.10.

[joalcava]

Any workaround for fedora users? I'm stuck with this.

[CRCinAU]

@joalcava - You can use the same as I suggested above - but change the yum commands for dnf.

[leecow]

Update on this: Debian, Ubuntu, OpenSUSE and SLES feeds should be up and running for Preview 2 with a fix dotnet-host fix to resolve the compat. Still having trouble with the rhel feed used for RHEL, Fedora, CentOS and Oracle need another team to do some surgery to the repo so will need to wait for Monday.

[bording]

@leecow I can confirm that the version of the dotnet-host preview package that gets installed no longer causes the crash (from the Ubuntu feed at least.)

However, I still don't understand why it's considered okay that installing a released version of the SDK or runtime installs preview packages in the first place. If I'm installing .NET Core on a production machine, I definitely don't want preview packages installed on it.

[steveharter]

Closing this for the original issue with the logging to stderr.

There still is the question above around why a release SDK installs the preview host.

[bording]

There still is the question above around why a release SDK installs the preview host.

@steveharter Is there an issue for this that I can follow?