This article is contributed. See the original author and article here.
Summary
The problem in this case was, somehow, being caused by the customer’s App Service having the .NET Core 3.1 runtime installed via Site Extension, instead of using the built-in runtime that comes with App Services.
The issue resolved when the Site Extension was removed, and the App Service was stopped and re-started.
Deeper Dive into the Data
This issue showed different symptoms depending on whether the ASP.NET Core app was running in-process or out-of-process.
In-Process
In-process, the symptom was a 500.30 In-Process Start Failure with error code 8007023e. This exception code means “unhandled exception.” Viewing the eventlog.xml in the App Service via Kudu came up with this couplet of events every time:
1018
1
0
Keywords
-1368025656
Application
[redacted]
Application ‘/LM/W3SVC/1365716517/ROOT’ with physical root ‘C:homesitewwwroot’ hit unexpected managed exception, exception code = ‘0xc0000005’. Please check the stderr logs for more information.
Process Id: 4236.
File Version: 13.1.22230.29. Description: IIS ASP.NET Core Module V2 Request Handler. Commit: 21d42143378ad6cc4bcbaebfda5f3acddf13aa47
…
Application ‘/LM/W3SVC/1365716517/ROOT’ with physical root ‘C:homesitewwwroot’ failed to load coreclr. Exception message: CLR worker thread exited prematurely
Process Id: 4236.
File Version: 13.1.22230.29. Description: IIS ASP.NET Core Module V2 Request Handler. Commit: 21d42143378ad6cc4bcbaebfda5f3acddf13aa47
It seems CoreCLR was trying to load and failed with a native access violation exception (c0000005). Very odd. We did not get a dump of this but I wish we had.
Out-of-Process
When switching the app to run out-of-process, we encountered a different error. This is from the eventlog.xml:
…
Application ‘/LM/W3SVC/1365716517/ROOT’ with physical root ‘C:homesitewwwroot’ failed to start process with commandline ‘”dotnet” .[redacted].dll’ with multiple retries. Failed to bind to port ‘31490’. First 30KB characters of captured stdout and stderr logs from multiple retries:
Process Id: 7032.
File Version: 13.1.22287.31. Description: IIS ASP.NET Core Module V2 Request Handler. Commit: fbe05294ac5c88be848b4d57d60cb2657874da9b
Nothing really useful there.
We enabled AspNetCoreModule’s Enhanced Diagnostic Logging and saw that it was timing out while waiting for the app to report itself as started:
[aspnetcorev2_outofprocess.dll] Failed HRESULT returned: 0x8027025a at D:a_work1ssrcServersIISAspNetCoreModuleV2OutOfProcessRequestHandlerserverprocess.cpp:727
8027025a= E_APPLICATION_ACTIVATION_TIMED_OUT: The app didn’t start in the required time.
We also enabled the stdout log via the web.config and found the app had started just fine:
dbug: Microsoft.Extensions.Hosting.Internal.Host[1]
Hosting starting
…
dbug: Microsoft.AspNetCore.Server.Kestrel[0]
No listening endpoints were configured. Binding to http://localhost:5000 by default.
info: Microsoft.Hosting.Lifetime[0]
Now listening on: http://localhost:5000
…
info: Microsoft.Hosting.Lifetime[0]
Application started. Press Ctrl+C to shut down.
info: Microsoft.Hosting.Lifetime[0]
Hosting environment: Test2
info: Microsoft.Hosting.Lifetime[0]
Content root path: C:homesitewwwroot
dbug: Microsoft.Extensions.Hosting.Internal.Host[2]
Hosting started
Another Observation & Resolution
[aspnetcorev2.dll] Initializing logs for ‘C:homeSiteExtensionsAspNetCoreRuntime.3.1.x86ancmaspnetcorev2.dll‘. Process Id: 7632.. File Version: 13.1.22287.31. Description: IIS ASP.NET Core Module V2. Commit: fbe05294ac5c88be848b4d57d60cb2657874da9b.
[aspnetcorev2.dll] Initializing logs for ‘C:Program Files (x86)IISAsp.Net Core ModuleV2aspnetcorev2.dll‘. Process Id: 7496.. File Version: 13.1.19331.0. Description: IIS ASP.NET Core Module V2. Commit: 62eee6e6d21c95668a9e9529dce6562cc6c9f3bf.
Brought to you by Dr. Ware, Microsoft Office 365 Silver Partner, Charleston SC.
Recent Comments