Friday, June 17, 2016

SharePoint 2016 CU installation failure: "Exception: The upgraded database schema doesn't match the TargetSchema"

After installing CU 201606 for SharePoint 2016, "SharePoint 2016 Products Configuration Wizard" threw out error:

Database is in compatibility range and upgrade is recommended

In ULS log doesn't help much, and Windows Event Viewer tell us:

Failed to upgrade SharePoint Products.
An exception of type Microsoft.SharePoint.PostSetupConfiguration.PostSetupConfigurationTaskException was thrown.  Additional exception information: The upgrade command is invalid or a failure has been encountered.
Failed to upgrade SharePoint Products.
Microsoft.SharePoint.PostSetupConfiguration.PostSetupConfigurationTaskException: Exception of type 'Microsoft.SharePoint.PostSetupConfiguration.PostSetupConfigurationTaskException' was thrown.
   at Microsoft.SharePoint.PostSetupConfiguration.UpgradeTask.Run()
   at Microsoft.SharePoint.PostSetupConfiguration.TaskThread.ExecuteTask()

So I went to Central Admin, and then clicked "Upgrade and Migration" page:

It brings us here:


We can see that the content database "SP_Content_team80_tmp" caused the problem.

It's easy to solve the problem. Just run the PowerShell script below:

Upgrade-SPContentDatabase SP_Content_team80_tmp -NoB2BSiteUpgrade -Confirm:$false

Then, re-run "SharePoint 2016 Products Configuration Wizard".


Hope this trick save you some time   :-)

Friday, May 27, 2016

SharePoint Online - replace Server-side programming with Client-side programming

Finally, with the help of WebHooks, we can use Client-side programming to replace Server-side programming in SharePoint. ( )

But, does it really resolve the problem of customization on SharePoint Online sites?

In many cases, YES, but we need to be very careful. Because "data communication" is moved from RAM-RAM to Computer-Computer.

1. Hardware Latency

Latency of communication between different processes on the same machine, is totally different from the one between different machines. Let's check it here ( ). "Main memory reference" consumes around 100 ns, and "Round trip within same data center" takes around 500 us, which means the latter is 5000 times slower.

For external servers (not in the same data center), "Round trip" may take more than 30 ms. That's 300,000 times slower.

Caching doesn't help much in many cases.

2. Stability

Let's assume that all servers are in the same data center. Is the network in a data center as robust as the RAM on one computer?

3. Extra hardware overhead

How much work it needs to handle a web service request? Please check here for "IIS Architectures"

How much extra CPU, Memory Access, DISK IO, Network IO will be consumed for each request? Do we need to pay for that?

A data center may handle one million concurrent users easily, but, when one user open a customized page, it may cause many HTTP(s) requests by the JavaScript on that page. And, each triggered workflow instance may also submit many HTTP(s) requests!

4. Development and trouble shooting

To move a workflow activity from "Server Object model" to "Client object model", for me, it's painful.

SharePoint 2013 CSOM APIs are powerful, but, because there is one more layer, it's more complex. However, this post suggests to utilize mature third-party APIs instead of "reinventing wheels". I totally don't agree about that, because of "Reliability".

5. Reliability

If everything is on-premise, for a normal middle size enterprise, they may utilize 10,000 APIs(through assemblies) from 20 different software vendors. That's fine. Everything is fully tested before deploying to production servers. Any update/patch will also be fully tested on non-production servers.

But, if there are 10,000 APIs(through Web Services) from 20 different vendors, how can we keep the whole systems stable? If, on average, each API is upgraded/changed every 10 years, then there will be 3 APIs(Web Services) being changed every day. And not likely the changes can get fully tested on non-production environment first.

In general, the quality of Microsoft products is pretty good, but, how many times Microsoft recalled updates of their products? Can we expect the software/service/APIs from those 20 vendors are all as good as the one from Microsoft? How much we need to pay for these APIs every year?

In summary, we can move everything into cloud, just need to be cautious.

Wednesday, May 25, 2016

How to avoid top navigation menu item wraps

To support mobile view, SharePoint 2013 wraps menu item automatically. That's not user friendly to PC users.  As the screenshot below show, 7 items are displayed in 13 rows!

 Most of solutions recommend to overwrite CSS for element "ul.dynamic", such as here and here. But that requires uploading a css file and then add the CSS reference in master page. Is there any other place uses "ul.dynamic"? If yes, then all of them get affected.

Below is a tip so we can avoid modifying master page.

Just a dummy menu item.

Friday, May 20, 2016

"Invalid feature definition" error during SharePoint 2016 configuration

The error message is confusing, but easy to fix the problem: give the SQL instance more RAM.

It needs at least 3.5GB RAM.

Below is what I got when only gave SQL instance 2GB RAM.

05/20/2016 11:45:26  9  ERR                                    An exception of type System.Xml.Schema.XmlSchemaException was thrown.  Additional exception information: Feature definition with Id ca7bd552-10b1-4563-85b9-5ed1d39c962a failed validation, file 'fieldswss4.xml', line 68, character 9:
The 'ListInternal' attribute is not allowed.
System.Xml.Schema.XmlSchemaException: Feature definition with Id ca7bd552-10b1-4563-85b9-5ed1d39c962a failed validation, file 'fieldswss4.xml', line 68, character 9:
The 'ListInternal' attribute is not allowed. ---> System.Xml.Schema.XmlSchemaValidationException: The 'ListInternal' attribute is not allowed.
   --- End of inner exception stack trace ---
   at Microsoft.SharePoint.Administration.SPElementManifest.ElementXmlValidationCallBack(Object sender, ValidationEventArgs evtargs)
   at System.Xml.Schema.XmlSchemaValidator.SendValidationEvent(String code, String arg)
   at System.Xml.Schema.XmlSchemaValidator.ValidateAttribute(String lName, String ns, XmlValueGetter attributeValueGetter, String attributeStringValue, XmlSchemaInfo schemaInfo)
   at System.Xml.Schema.XmlSchemaValidator.ValidateAttribute(String localName, String namespaceUri, XmlValueGetter attributeValue, XmlSchemaInfo schemaInfo)
   at System.Xml.XsdValidatingReader.ValidateAttributes()
   at System.Xml.XsdValidatingReader.ProcessElementEvent()
   at System.Xml.XsdValidatingReader.Read()
   at System.Xml.XmlReader.MoveToContent()
   at System.Xml.XmlReader.IsStartElement()
   at Microsoft.SharePoint.Utilities.SPUtility.XsdValidateXml(XmlTextReader xmlStreamReader, String friendlyName, String pathXsdFile, String tagExpectedRootNode, Int32 desiredPathVersion, ValidationEventHandler xsdValEventHandler)
   at Microsoft.SharePoint.Administration.SPElementManifest.ValidateDefinition(String pathToFeatureAndElementManifestXsdFile)
   at Microsoft.SharePoint.Administration.SPFeatureDefinition.ValidateElementManifestXml(String pathToFeatureAndElementManifestXsdFile)
   at Microsoft.SharePoint.Administration.SPFeatureDefinition.ValidateDefinition(String pathToFeatureAndElementManifestXsdFile)
   at Microsoft.SharePoint.Administration.SPFeatureDefinitionCollection.AddCore(SPFeatureDefinition featdef, SPSite site, String solutionHash, Boolean fForce, Boolean fDoValidation, String pathToFeatureAndElementManifestXsdFile)
   at Microsoft.SharePoint.Administration.SPFarmFactory.EnsureOutOfBoxFeaturesInstalled(SPFarm farm, String[] rgsOutOfBoxFeatures, Int32 compatibilityLevel)
   at Microsoft.SharePoint.Administration.SPFarmFactory.CreateBasicServices(SPFarm farm)
   at Microsoft.SharePoint.Administration.SPFarmFactory.Create()
   at Microsoft.SharePoint.Administration.SPFarm.Create(SqlConnectionStringBuilder configurationDatabase, SqlConnectionStringBuilder administrationContentDatabase, SqlConnectionStringBuilder siteMapDatabase, IdentityType identityType, String farmUser, SecureString farmPassword, SecureString masterPassphrase)
   at Microsoft.SharePoint.PostSetupConfiguration.ConfigurationDatabaseTask.CreateOrConnectConfigDb()
   at Microsoft.SharePoint.PostSetupConfiguration.ConfigurationDatabaseTask.Run()
   at Microsoft.SharePoint.PostSetupConfiguration.TaskThread.ExecuteTask()

Tuesday, May 17, 2016

"DirectoryNotFoundException" from "Restore-SPSite"

There are a few possible causes of this exception, but the error message is confusing.  In my case, it's either about "cannot find directory" or "invalid data".  Below are the ones.

  • Error 1

This one is easy. As the error message shows, the SQL Server Instance needs more RAM.
During my test, it needs at least 3.5GB memory.

  • Error 2

The source and destination farm are with slightly different version.
We can get the version number through PowerShell script:


In the end, I use free tool "HxD Hex Editor" modified the version number in the site collection backup file, to make it identical to the target farm. Of course this should only be applied to non-production environment, and may cause unexpected issues.

Error 1 =========================================

Restore-SPSite : 0x80070003
At E:\EricFang\Scripts\Restore\Restore.intranet2013SIT.80.Main.ps1:30 char:1
+ Restore-SPSite -Identity $SiteUrl -Path $FilePath -DatabaseServer
$DatabaseServe ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (Microsoft.Share...dletRestoreSite:
   SPCmdletRestoreSite) [Restore-SPSite], DirectoryNotFoundException
    + FullyQualifiedErrorId : Microsoft.SharePoint.PowerShell.SPCmdletRestoreS

.Net SqlClient Data Provider: System.Data.SqlClient.SqlException: There is insufficient system memory in resource pool 'default' to run this query.
   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj)
   at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
   at System.Data.SqlClient.SqlDataReader.HasMoreRows()
   at System.Data.SqlClient.SqlDataReader.ReadInternal(Boolean setTimeout)
   at System.Data.SqlClient.SqlDataReader.NextResult()
   at Microsoft.ResourceManagement.Data.DataAccess.RetrieveWorkflowDataForHostActivator(Int16 hostId, Int16 pingIntervalSecs, Int32 activeHostedWorkflowDefinitionsSequenceNumber, Int16 workflowControlMessagesMaxPerMinute, Int16 requestRecoveryMaxPerMinute, Int16 requestCleanupMaxPerMinute, Boolean runRequestRecoveryScan, Boolean& doPolicyApplicationDispatch, ReadOnlyCollection`1& activeHostedWorkflowDefinitions, ReadOnlyCollection`1& workflowControlMessages, List`1& requestsToRedispatch)
The Execute method of job definition Microsoft.SharePoint.Administration.SPConfigurationRefreshJobDefinition (ID 20715ec8-eff1-459b-a5a6-f1facd58609e) threw an exception. More information is included below.

There is insufficient system memory in resource pool 'internal' to run this query.
A severe error occurred on the current command.  The results, if any, should be discarded.

Error 2 =========================================

Restore-SPSite : 0x80070003
At E:\EricFang\Scripts\Restore\Restore.intranet2013DEV.80.Main.ps1:30 char:1
+ Restore-SPSite -Identity $SiteUrl -Path $FilePath -DatabaseServer
$DatabaseServe ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (Microsoft.Share...dletRestoreSite:
   SPCmdletRestoreSite) [Restore-SPSite], DirectoryNotFoundException
    + FullyQualifiedErrorId : Microsoft.SharePoint.PowerShell.SPCmdletRestoreS


Thursday, May 12, 2016

Workflow Manager 1.0 and SharePoint 2013 server object model

(This post is for SharePoint On Premise.)

With the release of Workflow Manager 1.0, developers are encouraged to choose "Client side object model" (CSOM or JSON).  The reason is simple: make SharePoint farm more stable, and easy to migrate to SharePoint Online.

But, what's the drawbacks?

1. Not easy to convert existing SharePoint 2010 workflow assemblies to the new version.

2. Client side object model is not as powerful as Server object model.

3. Learning curve for developers.

4. To improve performance & decrease network latency, may need to consider extra caching module.

All right, if, for some reason, we want to utilize the powerful Workflow Manager 1.0 and SharePoint server object model, what should we do?

One obvious options is to build web services which are hosted on SharePoint server. Then Workflow Manager can call them to get the functionalities.

That's awkard. We have to build two parts for a workflow.

Can we use Server object model in workflow CodeActivity directly, under CodeActivityContext?

The answer is YES. And, it's not too hard.

1. Install and configure Workflow Manager 1.0 on a SharePoint Server.

2. Build the activity.

Need to think about the parameters. I prefer to add 4 parameters to each activity. As the name suggested, the last one will pass in the SPListItem URL and ItemID, so we can get SPWeb and SPListItem easily.

I didn't find a way to put log into workflow history from activity, so all information need to be recorded in parameter "ReturnDetailedInfo".

"ReturnCode" tells the workflow instance whether the activity succeed or fail.
"ReturnValue" return the result.

<Parameter Name="ReturnCode"  Type="System.String, mscorlib" Direction="Out" DesignerType="ParameterNames" Description="Workflow variable output" />
<Parameter Name="ReturnValue"  Type="System.String, mscorlib" Direction="Out" DesignerType="ParameterNames" Description="Workflow variable output" />
<Parameter Name="ReturnDetailedInfo"  Type="System.String, mscorlib" Direction="Out" DesignerType="ParameterNames" Description="Workflow variable output" />
<Parameter Name="CurrentItemUrl" Type="System.String, mscorlib" Direction="In" DesignerType="StringBuilder" />

3. Deploy the workflow (activity) assemblies and solution files to the SharePoint farm and the  workflow server.

Don't forget to create/modify "AllowedTypes.xml" in "C:\Program Files\Workflow Manager\1.0\Workflow\WFWebRoot\bin" and "C:\Program Files\Workflow Manager\1.0\Workflow\Artifacts"

Need to reboot windows service "Workflow Manager Backend" during the deployment.

Reference link: 

4. Make sure workflow service account has "SPDataAccess" rights of SharePoint Configuration database and the relevant content databases

$PrincipleName = "domainname\_WFServices"
Get-SPWebApplication | %{$_.GrantAccessToProcessIdentity($PrincipleName)}

5. Make sure workflow service account is in "WSS_WPG" local windows user group
Reboot the server after adding the service account to "WSS_WPG".

6. Confirm workflow service account has rights on c2WTS

RDP to the workflow server as the service account. The test can be done through c2WTS Tester

7. Restore the site collection to the target SharePoint farm


8. Make sure workflow service account has rights to access target site collection

We can build a X64 windows console program to test it.
RDP to the workflow server as the service account to test it.

9. Clean up Distributed Cache

Reboot all caching servers, or run the script below on all caching servers.
The script below come from here.

Get-SPServiceInstance | Where-Object { $_.TypeName -eq "Distributed Cache" } | Stop-SPServiceInstance -Confirm:$false | Out-Null
While (Get-SPServiceInstance | Where-Object { $_.TypeName -eq "Distributed Cache" -and $_.Status -ne "Disabled" }) {
    Start-Sleep -Seconds 15

Get-SPServiceInstance | Where-Object { $_.TypeName -eq "Distributed Cache" } | Start-SPServiceInstance | Out-Null
While (Get-SPServiceInstance | Where-Object { $_.TypeName -eq "Distributed Cache" -and $_.Status -ne "Online" }) {
    Start-Sleep -Seconds 15

10. Build and publish the workflows through SharePoint Designer;

11. IISReset on Web Front End servers

Below is the sample code.

        protected override void Execute(CodeActivityContext context)
            string strTmp = string.Empty;

            SPList objSPList = null;
            SPListItem objSPListItem = null;

                this._ArgCurrentItemUrl = this.CurrentItemUrl.Get(context);
                Uri UriCurrentItemUrl = new Uri(_ArgCurrentItemUrl);
                NameValueCollection queryStrings = HttpUtility.ParseQueryString(UriCurrentItemUrl.Query);
                if (queryStrings["ID"] == null)
                    strTmp = string.Format("list '{0}' doesn't exist!", _CurrentListName);
                this._ItemID = int.Parse(queryStrings.Get("ID"));

                for (int i = 0; i < UriCurrentItemUrl.Segments.Length; i++)
                    if (UriCurrentItemUrl.Segments[i].Equals(@"Lists/", StringComparison.InvariantCultureIgnoreCase))
                        this._CurrentListName = UriCurrentItemUrl.Segments[i + 1];
                        this._CurrentListName = this._CurrentListName.Substring(0, this._CurrentListName.Length - 1);
                using (SPSite objSPSite = new SPSite(this._ArgCurrentItemUrl))
                    using (SPWeb objSPWeb = objSPSite.OpenWeb())
                        this._CurrentWebUrl = objSPWeb.Url;
                        objSPList = objSPWeb.Lists.TryGetList(this._CurrentListName);
                        if (objSPList == null)
                            objSPList = objSPWeb.Lists[this._CurrentListName];
                            if (objSPList == null)
                                strTmp = string.Format("list '{0}' doesn't exist in web '{1}'!", this._CurrentListName, this._CurrentWebUrl);
                                throw new Exception(strTmp);
                            this._CurrentListName = objSPList.Title;
                        objSPListItem = objSPList.GetItemById(_ItemID);
                        if (objSPListItem == null)
                            strTmp = string.Format("list item '{0}' doesn't exist in list '{1}', in web '{2}'!",
                                this._ItemID, this._CurrentListName, this._CurrentWebUrl);

                this._ArgSQLConnectionString = this.SQLConnectionString.Get(context);
                this._ArgColumnMappingXml = this.ColumnMappingXml.Get(context);
                this._ArgSQLSelectCommand = this.SQLSelectCommand.Get(context);

            catch (Exception ex)
                strTmp = string.Format(@"Execute(), ex.Message={0}", ex.Message);
                strTmp = string.Format(@"Execute(), ex.StackTrace={0}", ex.StackTrace);
                this.ReturnCode.Set(context, this._ArgReturnCode);
                this.ReturnValue.Set(context, this._ArgReturnValue);
                this.ReturnDetailedInfo.Set(context, this._ArgReturnDetailedInfo);

Monday, May 9, 2016

The future of SharePoint development

SharePoint 2016 is finally released in GA. The astonishing thing is "SharePoint Framework".

It seems Microsoft stopped pushing "Addins (APPS)", and invented another development model. I completely understand why some SharePoint experts not happy about that.

I am OK about it, because I didn't spend much time on "Addins" model, and no plan to study "SharePoint Framework".  :-)

I don't know why Microsoft keep building these complex development model. This is just like how Microsoft Vista handle the security problem: pop up a prompt window, let users decide whether accept the potential threat. If the user choose the wrong one......Microsoft doesn't need to be responsible for that, right?

Same problem with full trust farm solution. There are a lot of poor quality fully trusted code, and Microsoft's solution is simple: ban all of them. If users want the functionality, they have to run it in other application servers. If the other application servers crashed because of the poor quality code......Microsoft doesn't need to be responsible for that, right?

Client-side application is very important. I have no doubt about that. But, use it to replace server-side application? In many cases, that make things very complicated. And complexity make poor quality code much worse! And, it decreases the productivity of development, A LOT! (Think about latency, caching, stability, network bandwidth, etc.)

Kicking out the problem, is not really a solution.

What's the real solution?  I have some ideas here.

Split the development into two categories: server-side and client-side.

For "server-side", put each fully trusted solution into a dedicated "container". If the code crashed, the container will crash and reset, and it doesn't affect the other parts of SharePoint. Many of the SharePoint OOTB service instances should also run in "container".

For "client-side", it's actually JavaScript running in something like Content Editor Web Part, with the help of OOTB toolbox/library. (Users can build TypeScript then turn it into JavaScript, of course)

Can we get that in SharePoint 2019? Let's wait and see.

Any comments are welcome.

PS: What is going to replace InfoPath?!

[update, 2016-05-20]

Don't get me wrong. SharePoint 2016 is good, just not so friendly to developers.  :-)