SharePoint 2013 April 2013 CU installation trouble shooting

SharePoint 2013 April 2013 CU is released!  I couldn't wait to try it.

In one farm, it went very well. But in another one, the upgrade failed.

Below is what I found in windows events log.

Failed to upgrade SharePoint Products.
An exception of type Microsoft.SharePoint.Upgrade.SPUpgradeException was thrown.  Additional exception information: Upgrade completed with errors.  Review the upgrade log file located in C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\LOGS\Upgrade-20130430-111738-752.log.  The number of errors and warnings is listed at the end of the upgrade log file.
Microsoft.SharePoint.Upgrade.SPUpgradeException: Upgrade completed with errors.  Review the upgrade log file located in C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\LOGS\Upgrade-20130430-111738-752.log.  The number of errors and warnings is listed at the end of the upgrade log file.
   at Microsoft.SharePoint.Upgrade.SPUpgradeSession.CheckPoint()
   at Microsoft.SharePoint.Upgrade.SPUpgradeSession.LogEnd()
   at Microsoft.SharePoint.PostSetupConfiguration.UpgradeTask.Run()
   at Microsoft.SharePoint.PostSetupConfiguration.TaskThread.ExecuteTask()

Configuration of SharePoint Products failed.  Configuration must be performed in order for this product to operate properly.  To diagnose the problem, review the extended error information located at C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\LOGS\PSCDiagnostics_4_30_2013_11_17_34_737_955156400.log, fix the problem, and run this configuration wizard again.

In the upgrade log file, I found the error messages:

04/30/2013 10:55:26.80    OWSTIMER (0x10F8)    0x16FC    SharePoint Foundation Upgrade    SPUpgradeSession    aj0ur    ERROR    Upgrade Timer job is exiting due to exception: Microsoft.SharePoint.Upgrade.SPUpgradeException: Upgrade completed with errors.  Review the upgrade log file located in C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\LOGS\Upgrade-20130430-104811-344.log.  The number of errors and warnings is listed at the end of the upgrade log file.     at Microsoft.SharePoint.Upgrade.SPUpgradeSession.CheckPoint()     at Microsoft.SharePoint.Upgrade.SPUpgradeSession.LogEnd()     at Microsoft.SharePoint.Administration.SPUpgradeJobDefinition.Execute(Guid targetInstanceId)    91a9169c-2c55-d039-643e-29605165d88b
04/30/2013 10:55:26.88    OWSTIMER (0x10F8)    0x16FC    SharePoint Foundation Upgrade    SPManager    ajxo1    DEBUG    Removing exclusive upgrade regkey by setting the mode to none    91a9169c-2c55-d039-643e-29605165d88b
04/30/2013 10:55:26.88    OWSTIMER (0x10F8)    0x16FC    SharePoint Foundation Upgrade    SPUpgradeSession    ajxl4    INFO    ======================================================================================    91a9169c-2c55-d039-643e-29605165d88b
04/30/2013 10:55:26.88    OWSTIMER (0x10F8)    0x16FC    SharePoint Foundation Upgrade    SPUpgradeSession    ajxl5    INFO    Upgrade session failed. 4 errors and 0 warnings encountered.    91a9169c-2c55-d039-643e-29605165d88b
04/30/2013 10:55:26.88    OWSTIMER (0x10F8)    0x16FC    SharePoint Foundation Upgrade    SPUpgradeSession    ajxl6    INFO    Look for possible causes for upgrade issues by searching [ERROR] and [WARNING] strings in the upgrade log file. Refer to "http://go.microsoft.com/fwlink/?LinkId=259653" for more information about how to recover from upgrade failures.    91a9169c-2c55-d039-643e-29605165d88b
04/30/2013 10:55:26.88    OWSTIMER (0x10F8)    0x16FC    SharePoint Foundation Upgrade    SPUpgradeSession    ajxl7    INFO    ======================================================================================    91a9169c-2c55-d039-643e-29605165d88b

In PSCDiagnostics log file:

04/30/2013 11:15:31  1  ERR          Failed to upgrade SharePoint Products.
An exception of type Microsoft.SharePoint.Upgrade.SPUpgradeException was thrown.  Additional exception information: Upgrade completed with errors.  Review the upgrade log file located in C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\LOGS\Upgrade-20130430-111314-494.log.  The number of errors and warnings is listed at the end of the upgrade log file.
Microsoft.SharePoint.Upgrade.SPUpgradeException: Upgrade completed with errors.  Review the upgrade log file located in C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\LOGS\Upgrade-20130430-111314-494.log.  The number of errors and warnings is listed at the end of the upgrade log file.
   at Microsoft.SharePoint.Upgrade.SPUpgradeSession.CheckPoint()
   at Microsoft.SharePoint.Upgrade.SPUpgradeSession.LogEnd()
   at Microsoft.SharePoint.PostSetupConfiguration.UpgradeTask.Run()
   at Microsoft.SharePoint.PostSetupConfiguration.TaskThread.ExecuteTask()

Not sure what happened. The error message is not really helpful.  So I tried to upgrade it one more time manually through command "psconfig.exe -cmd upgrade -inplace b2b -wait -force".

This time, in PSCDiagnostics log file, finally there was something useful.


04/30/2013 11:18:39.07    PSCONFIG (0x0A00)    0x0E50    SharePoint Foundation Upgrade    SPUpgradeSiteSession    ajxn2    DEBUG    UPGRADE PERCENTAGE: 100.00000000% done, Total Elapsed Time 00:00:00.8281472.    00000000-0000-0000-0000-000000000000
04/30/2013 11:18:39.07    PSCONFIG (0x0A00)    0x0E50    SharePoint Foundation Upgrade    SPUpgradeSiteSession    aloox    DEBUG    End processing children of [SPSite Url=http://hvsp2013:8080/personal/pw2__spsetup] for upgrade.    00000000-0000-0000-0000-000000000000
04/30/2013 11:18:39.07    PSCONFIG (0x0A00)    0x0E50    SharePoint Foundation Upgrade    SPUpgradeSiteSession    alooy    DEBUG    Exiting upgrade for [SPSite Url=http://hvsp2013:8080/personal/pw2__spsetup].  Elapsed Time=[00:00:00]    00000000-0000-0000-0000-000000000000
04/30/2013 11:18:39.07    PSCONFIG (0x0A00)    0x0E50    SharePoint Foundation Upgrade    SPManager    ajxhh    INFO    Finished upgrading [SPSite Url=http://hvsp2013:8080/personal/pw2__spsetup].    00000000-0000-0000-0000-000000000000
04/30/2013 11:18:39.07    PSCONFIG (0x0A00)    0x0E50    SharePoint Foundation Upgrade    SPUpgradeSiteSession    ajxl4    INFO   

Not sure which feature caused the upgrade failure. As a quick fix, I deleted the personal site collection of user "pw2\_spsetup".

Then, "psconfig.exe -cmd upgrade -inplace b2b -wait -force" upgraded the whole farm successfully.

Error: The site collection could not be restored

During the daily work, I always create a new site collections in Dev or Test environment first.  If end users are happy about it, then I take a backup and restore it to Production server.

However, a few days ago, I start to get the error below during the site collection restoration.


VbackupRestore::RestoreSite, failed HR detected: -2147023893

The site collection could not be restored. If this problem persists, please make sure the content databases are available and have sufficient free space.0x81020099

Error Category: InvalidData    Target Object  Microsoft.SharePoint.PowerShell.SPCmdletRestoreSite  Details  NULL  RecommendedAction NULL

Obviously there are some objects in the destination database which have the same GUID as the objects in backup file. But, how can a new site collection has objects conflict with existing site collections?

Then I recalled that I created a customized site template for new site collections. I thought that new GUIDs were created for all objects when we choose old site template --- I was wrong.  Site definition is not site template. Some object GUIDs are not been replaced by new GUIDs!

So, no choice. I have to recreate the site collection from scratch in Dev environment, then deploy it to Production. Any way, if there is no coding, that's normally easy and simple.

Error: Violation of UNIQUE KEY constraint 'Docs_IdLevelUnique'

When trying to restore a site collection to a content database through PowerShell command "Restore-SPSite", I got the error message below.


System.Data.SqlClient.SqlException: Violation of UNIQUE KEY constraint 'Docs_IdLevelUnique'. Cannot insert duplicate key in object 'dbo.AllDocs'. The duplicate key value is (bd416c52-fd3f-477e-b067-5a20ff243896, 1).  The statement has been terminated.    
 at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)    
 at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj)    
 at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)    
 at System.Data.SqlClient.SqlDataReader.ConsumeMetaData()    
 at System.Data.SqlClient.SqlDataReader.get_MetaData()    
 at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)    
 at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async)    
 at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result)    
 at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method)    
 at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior, String method)    
 at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior)    
 at Microsoft.SharePoint.Utilities.SqlSession.ExecuteReader(SqlCommand command, CommandBehavior behavior, SqlQueryData monitoringData, Boolean retryForDeadLock)

at Microsoft.SharePoint.SPSqlClient.ExecuteQueryInternal(Boolean retryfordeadlock)    
 at Microsoft.SharePoint.SPSqlClient.ExecuteQuery(Boolean retryfordeadlock)    
 at Microsoft.SharePoint.Library.SPRequestInternalClass.RestoreSite(String bstrUrl, String bstrFileName, Boolean bCurrentServerADMode, Int32 major, Int32 minor, Int32 build, Int32 revision)    
 at Microsoft.SharePoint.Library.SPRequest.RestoreSite(String bstrUrl, String bstrFileName, Boolean bCurrentServerADMode, Int32 major, Int32 minor, Int32 build, Int32 revision)    
 at Microsoft.SharePoint.SPSite.Restore(String filename, Boolean isADMode)    
 at Microsoft.SharePoint.Administration.SPSiteCollection.Restore(String strSiteUrl, String strFilename, Boolean bOverwrite, Boolean bGradualDelete, Boolean hostHeaderAsSiteName)    
 at Microsoft.SharePoint.PowerShell.SPCmdletRestoreSite.InternalProcessRecord()    
 at Microsoft.SharePoint.PowerShell.SPCmdlet.ProcessRecord()    
 at System.Management.Automation.CommandProcessor.ProcessRecord()    
 at System.Management.Automation.CommandProcessorBase.DoExecute()    
 at System.Management.Automation.Internal.PipelineProcessor.SynchronousExecuteEnumerate(Object input, Hashtable errorResults, Boolean enumerate)    
 at System.Management.Automation.PipelineNode.Execute(Array input, Pipe outputPipe, ArrayList& resultList, ExecutionContext context)    
 at System.Management.Automation.StatementListNode.ExecuteStatement(ParseTreeNode statement, Array input, Pipe outputPipe, ArrayList& resultList, ExecutionContext context)    
 at System.Management.Automation.StatementListNode.Execute(Array input, Pipe outputPipe, ArrayList& resultList, ExecutionContext context)    
 at System.Management.Automation.ParseTreeNode.Execute(Array input, Pipe outputPipe, ExecutionContext context)    
 at System.Management.Automation.ScriptCommandProcessor.ExecuteWithCatch(ParseTreeNode ptn, Array inputToProcess)    
 at System.Management.Automation.ScriptCommandProcessor.RunClause(ParseTreeNode clause, Object dollarUnderbar, Object inputToProcess)    
 at System.Management.Automation.CommandProcessorBase.DoComplete()    
 at System.Management.Automation.Internal.PipelineProcessor.SynchronousExecuteEnumerate(Object input, Hashtable errorResults, Boolean enumerate)    
 at System.Management.Automation.Runspaces.LocalPipeline.InvokeHelper()    
 at System.Management.Automation.Runspaces.LocalPipeline.InvokeThreadProc()    
 at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)    
 at System.Threading.ThreadHelper.ThreadStart()

SqlError: 'Violation of UNIQUE KEY constraint 'Docs_IdLevelUnique'. Cannot insert duplicate key in object 'dbo.AllDocs'. The duplicate key value is (bd416c52-fd3f-477e-b067-5a20ff243896, 1).'    Source: '.Net SqlClient Data Provider' Number: 2627 State: 1 Class: 14 Procedure: '' LineNumber: 1 Server: 'DB_SharepointQA\SHAREPOINT'

SqlError: 'The statement has been terminated.'    Source: '.Net SqlClient Data Provider' Number: 3621 State: 0 Class: 0 Procedure: '' LineNumber: 1 Server: 'DB_SharepointQA\SHAREPOINT'

I know this is because I deleted the same site collection before, and some data are actually still there in the content database. However, it seems there is no easy way to fix it! Thanks for the post here, I realized that the only solution might be moving site collections to another content database.

Below is the script. I recreated the content database because want to keep the same database name.


New-SPContentDatabase SP_Content_team80_new  -WebApplication http://sitename

Get-SPSite -ContentDatabase SP_Content_team80 | Move-SPSite -DestinationDatabase SP_Content_team80_new

Get-SPContentDatabase -Identity "SP_Content_team80" | Remove-SPContentDatabase

New-SPContentDatabase SP_Content_team80  -WebApplication http://sitename

Get-SPSite -ContentDatabase SP_Content_team80_new | Move-SPSite -DestinationDatabase SP_Content_team80

Get-SPContentDatabase -Identity "SP_Content_team80_new" | Remove-SPContentDatabase

Can we use "Gradual Site Delete timer job" to do the job?  Not sure. I didn't specify "GradualDelete" parameter when deleting the site collection, so it should not work in theory. Any way, I forgot to try that before moving all those site collections.

Free tool to browse the scope hierarchy of a Workflow Manager 1.0 installation

I am surprised that there is no out-of-the-box tool to view objects details of a Workflow Manager 1.0 installation. Lucky that Dave Cliffe from Microsoft built Workflow Resource Browser Tool to show us the scope hierarchy, and view information about activities, workflows, security policies and configuration under the currently selected scope.

After downloading the source code, you need to compile it and build the assembly through Visual Studio 2012.  Or you can download Microsoft.Workflow.Explorer.bin.rar directly.

Workflow Service Address is the only information we need.

Home UI.

Activities.

Activity details ( XAML ).

Security policy.

On-premise windows workflow server configuration with SharePoint 2013

[Environment background] Service Bus 1.0 and Workflow Manager 1.0 are installed on the same virtual machine.  SharePoint server 2013 and SQL Server are installed on another virtual machine.  They are in the same domain.

After registering the new workflow server with SharePoint through PowerShell command:

Register-SPWorkflowService –SPSite "http://hvApp2012.pw3.local/" –WorkflowHostUri "https://hvWF2012.pw3.local:12290" –AllowOAuthHttp

I can see "SharePoint 2013 Workflow" platform type when trying to create declarative workflow in SharePoint Designer 2013.

I can build the workflow. However, when trying to publish it, I got the error below.

System.InvalidOperationException: Operation failed with error Microsoft.Workflow.Client.ScopeNotFoundException: Scope '/SharePoint/default' was not found.  HTTP headers received from the server - ActivityId: f8958d89-6946-4e76-b502-b8669b68188e. NodeId: HVWF2012. Scope: /SharePoint/default/bc770e6d-619f-4d8d-9564-a97222b1afbc. Client ActivityId : 35ee0d9c-20f3-b0fb-0015-1deca478baeb. ---> System.Net.WebException: The remote server returned an error: (404) Not Found.
   at Microsoft.Workflow.Com

(In an separate test environment, the workflow publishing is fine. But the workflow always cancels itself automatically after getting triggered)

It seems we need to specify workflow "scope" for the SharePoint site. But I cannot find it from MSDN sample code (Obviously, they really, really want everyone to move to Azure)

The PowerShell commands of "Workflow Manager 1.0" are here.

So I created a new scope as workflow manager "pw3\_wfManager":

New-WFScope -ScopeUri https://hvWF2012.pw3.local:12290/SP2013.PW3.hvApp2012.80

We can see the result:

$mycredentials = Get-Credential("pw3\_wfManager")
Get-WFScope -ScopeUri https://hvWF2012.pw3.local:12290/SP2013.PW3.hvApp2012.80 -Credential $mycredentials

Then grant the access rights of this scope to all users:

$SecurityConfig = New-Object Microsoft.Workflow.Client.Security.WindowsSecurityConfiguration("All Users")
$SecurityConfig.WorkflowAdminGroupName = "Users"
Set-WFScopeSecurity -ScopeUri https://hvWF2012.pw3.local:12290/SP2013.PW3.hvApp2012.80 -Credential $mycredentials -SecurityConfiguration $SecurityConfig

The reason that I use "WindowsSecurityConfiguration" instead of "OAuthWrapSecurityConfiguration", is because I can see the root scope information when open workflow server site https://hvwf2012.pw3.local:12290/ :

- <ScopeInfo xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.microsoft.com/workflow/2012/xaml/activities">
  <ChildScopeCount>1</ChildScopeCount> 
  <DefaultWorkflowConfiguration /> 
  <Description>Root Scope</Description> 
  <LastModified>2013-03-27T05:44:01.13</LastModified> 
  <LastRevised>2013-03-27T05:44:01.13</LastRevised> 
  <Path>/</Path> 
- <SecurityConfigurations>
- <ScopedSecurityConfiguration i:type="WindowsSecurityConfiguration">
  <Name>Microsoft.Workflow.Management.Security.WindowsSecurityConfiguration</Name> 
  <WorkflowAdminGroupName>BUILTIN\Administrators</WorkflowAdminGroupName> 
  </ScopedSecurityConfiguration>
  </SecurityConfigurations>
  <Status>Active</Status> 
  </ScopeInfo>

If you got error message:

Authentication Failed. Valid credentials must be provided for one of the following protocols: Negotiate.

You can turn off "Anonymous Authentication" from workflow server IIS site settings. But don't forget to change it back after changing the scope security. Or else you will get "access denied" error when registering the workflow service.

After that, we can go back to SharePoint server to register the workflow service.

Register-SPWorkflowService –SPSite "http://hvApp2012.pw3.local/" –WorkflowHostUri "https://hvWF2012.pw3.local:12290" -Force -ScopeName "SP2013.PW3.hvApp2012.80"

How to reset / fix Document ID?

When we restore a site collection to another site collection which is with different Document ID prefix, sometimes we still get the initial Document ID and Document ID URL, which is incorrect. This problem also happens when user use "copy and paste" through "Windows File Explorer" to generate new file.

I know the relevant field names are "_dlc_DocId" and "_dlc_DocIdUrl", so at first, I tried to use C# or PowerShell to update the Document ID value with no luck.

Then I reset the Document ID prefix in "Document ID Settings", and ran "Document ID assignment job" manually from Central Admin, Job Definitions page. Still no luck.

After that, I downloaded the file to local hard drive. It's a MS Word file, so I opened it through MS Word, and did find the property "Document ID Value". However it's a read-only property, we cannot modify it directly.

The rest steps are quite simple: remove all document properties, then upload the file to SharePoint to overwrite the previous one.

Below is how I did it in MS WORD 2010.

If the Document ID is not changed, you may need to reset all Document ID of the current site collection.

Then run "Document ID assignment job" for relevant web application manually.