Intro
Our story begins with fresh installation of Windows Server 2019 and a client wanting me to help them out install a SharePoint 2019 farm. Since I’ve done these hundreds of times, I thought this one is going to be easy, but no….
SharePoint in Trial Mode
After a fresh SharePoint farm was configured, each and every time, SharePoint would go into trial mode. We tried to reinstall the system a couple of times before I dug into it and tried troubleshooting this issue. First thing that I tried is of course to try and get a clue from SharePoint logs, but apart from getting the info that the ValidateLicense() method failed, nothing really popped out.
Microsoft.Office.Server.ProductExpiredException: The trial period for this product has expired or Secure Store Shared Service is not supported for this SKU.
at Microsoft.Office.SecureStoreService.Server.SecureStoreServiceApplicationProxy.ValidateLicense()
at Microsoft.Office.SecureStoreService.Server.SecureStoreServiceApplicationProxy.Execute[T](String operationName, Boolean validateCanary, ExecuteDelegate`1 operation)
at Microsoft.Office.SecureStoreService.Server.SecureStoreServiceApplicationProxy.IsMasterSecretKeyPopulated()
at Microsoft.Office.SharePoint.ClientExtensions.SecureStoreAdministration.SSSAdminHelper.EnsurePrerequisite(SecureStoreServiceApplicationProxy proxy, String& errorMessage)
at Microsoft.Office.SharePoint.ClientExtensions.SecureStoreAdministration.ManageSSSvcApplication.InitializeGridView()
at Microsoft.Office.SharePoint.ClientExtensions.SecureStoreAdministration.ManageSSSvcApplication.OnLoad(EventArgs e)
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
Since I ran out of options, I opened a case with a Microsoft support, where they gave me a tool called “GetLicenseType2019.exe”. This tool was really useful, because it gave me two important clues.
The first clue was that the dll responsible for doing the license validation is “OfficeServerSettings.dll” and the other one being “You can’t access this shared folder because your organization’s security policies block unauthenticated guest access.”. When googling around for the second clue, I found out that this has something to do with SMB being disabled in Windows Server 2019 and that there is a way of enabling guest logins for SMB. So, I thought that maybe if I do a network trace while running the tool, I will get some clues.
Just running a network trace for SMB did not get me any useful results, but one thing I did notice was that all the time one IP address was popping out.
So then I thought, OK, if the error was SMB related and this is disabled, maybe I should modify the local policy and enable »Enable insecure guest logins«.
With this on and again running the tool while doing the network capture, the tool was properly working and the network trace showed the same IP address again, but this time also some network paths were shown in the network trace.
That is odd, I thought. At this time, I also got a senior escalation engineer interested in the case and we started digging in together. And if you want to get a proper insight, you have to open the PROCMON tool.
PROCMON
At that time, we knew 2 things, that OfficeServerSettings.DLL is causing the issue and that, for some reason, a network path is involved in the license check. I again disabled the “Enable insecure guest logins” setting.
Firing up procmon and then setting a path-based filter for the OfficeServerSettings, gave us the clue about what is happening behind the scenes.
SharePoint is going through some list of locations and is trying to find the OfficeServerSettings.dll and, for some reason, it was also trying to access this dll through a network path.
At that time, I got the information that SharePoint is using the environment variables to find the proper location of the dll file and also while searching the registry, I found this value in the path settings. Since apparently not all exceptions are properly handled in the license validation code, when it got the network path SMB access denied exception, it stopped searching other directories and threw an exception of SharePoint not having a valid license.
Looking at the Path variable, we found the path being in front of the location where the dll is stored.
We then moved the path to the bottom of the list and tried running the tool again. Sure enough, the tool was working and displaying the proper licensing information.
But, when opening a SharePoint Central administration page, it was still displaying that it is in trial mode. We did an IISReset, nothing helped, SharePoint still being stubborn and being in trial mode.
Process Explorer
Strange, we thought, and then fired up Process explorer and checked what kind of data the w3wp.exe process has. We again found the path in question being in front of the actual DLL location and another useful piece of information from the process explorer is the process tree where noticing that the w3wp process is a child process of the svchost.exe and w3wp.exe is inheriting data from that process.
So even if you do an IISreset, it does not help, because the data in svchost in not updated. Since svchost can only be populated when system starts, we did a server reboot and sure enough, everything started working as expected.
Thank you sysinternals tools!!
