Friday, January 9, 2015

ProfileSubtypeProperty, CoreProperty, ProfileTypeProperty and syncConnection PropertyMapping

Some attributes belong to ProfileSubtypeProperty object, some belong to CoreProperty, and some belong to ProfileTypeProperty.

It would be nice to see which one belongs to which one.

Below is the script I built to help.

======== list user profile properties and syncConnection PropertyMapping ========

# https://gallery.technet.microsoft.com/office/SP2010-PowerShell-to-0fda04f7
# http://phadnisblog.com/2012/03/13/how-do-i-add-custom-user-profile-properties-with-powershell/
# http://sharepoint.stackexchange.com/questions/56972/delete-sharepoint-userprofile-property-using-powershell
# http://msdn.microsoft.com/en-us/library/office/ms496130(v=office.14).aspx
# http://blogs.msdn.com/b/chandru/archive/2011/09/17/powershell-script-to-create-a-new-user-profile-property.aspx
# http://sp2007hut.wordpress.com/2012/02/11/creating-sharepoint-2010-upa-properties-via-powershell/

cls

$connectionName="user profile sync connection Name"

function Get-SPServiceContext([Microsoft.SharePoint.Administration.SPServiceApplication] $profileApp)
{
    $profileApp = @(Get-SPServiceApplication | ? {$_.TypeName -eq "User Profile Service Application"})[0]
    return [Microsoft.SharePoint.SPServiceContext]::GetContext($profileApp.ServiceApplicationProxyGroup, [Microsoft.SharePoint.SPSiteSubscriptionIdentifier]::Default)
}

$serviceContext = Get-SPServiceContext
$userProfileConfigManager = New-Object Microsoft.Office.Server.UserProfiles.UserProfileConfigManager($serviceContext)

Write-Host "PropertyMapping, connectionName="$connectionName -ForegroundColor Yellow
$syncConnectionManager = $userProfileConfigManager.ConnectionManager[$connectionName]
$syncConnectionPropertyMapping = $syncConnectionManager.PropertyMapping
foreach ($itemPropertyMapping in $syncConnectionPropertyMapping.GetEnumerator())
{
    #if ($itemPropertyMapping.ProfileProperty.Name -NotMatch "SPS")
    #{
        #$itemPropertyMapping | fl *
        $itemPropertyMapping | select-object @{Name="Property Name"; Expression={$_.ProfileProperty.Name}}, DataSourcePropertyName, IsImport, IsExport
        break
    #}
}

$userProfilePropertyManager = $userProfileConfigManager.ProfilePropertyManager

$defaultSubType = [Microsoft.Office.Server.UserProfiles.ProfileSubtypeManager]::GetDefaultProfileName([Microsoft.Office.Server.UserProfiles.ProfileType]::User)

$coreProperties = $userProfilePropertyManager.GetCoreProperties()
Write-Host "coreProperties.Count="$coreProperties.Count -ForegroundColor Yellow

$userProfileTypeProperties = $userProfilePropertyManager.GetProfileTypeProperties([Microsoft.Office.Server.UserProfiles.ProfileType]::User)
$userProfileSubTypeManager = [Microsoft.Office.Server.UserProfiles.ProfileSubTypeManager]::Get($serviceContext)
$userProfileSubtype = $global:userProfileSubTypeManager.GetProfileSubtype($defaultSubType)

Write-Host "userProfileTypeProperties.Count="$userProfileTypeProperties.Count -ForegroundColor Yellow
Write-Host "userProfileSubtype.PropertiesWithSection.Count="$userProfileSubtype.PropertiesWithSection.Count -ForegroundColor Yellow
Write-Host "userProfileSubtype.Properties.Count="$userProfileSubtype.Properties.Count -ForegroundColor Yellow

Write-Host "Display first coreProperty" -ForegroundColor Green
foreach ($profileCoreProperty in $coreProperties)
{
    $profileCoreProperty | fl *
    break
}

foreach ($profileSubtypeProperty in $userProfileSubtype.Properties)
{
Write-Host "Display first SubtypeProperty" -ForegroundColor Green
    $profileSubtypeProperty | fl *
Write-Host "Display first TypeProperty" -ForegroundColor Green
    $profileSubtypeProperty.TypeProperty | fl *
    break
}

Write-Host "Display first coreProperty with no TypeProperty" -ForegroundColor Green
foreach ($profileCoreProperty in $coreProperties)
{
    $profileSubtypeProperty = $userProfileSubtype.Properties.GetPropertyByName($profileCoreProperty.Name)
    if ($profileSubtypeProperty -eq $null)
    {
        $profileCoreProperty | fl *
        break
    }
}





Wednesday, January 7, 2015

SharePoint 2010 installation error: The specified value for the LocStringId parameter is outside the bounds of this enum

After re-installing Windows Server 2008 R2 SP1, I downloaded SharePoint 2010 with SP2 installation media from MSDN ( "en_sharepoint_server_2010_with_service_pack_2_x64_dvd_4178583.iso"), then tried to install a new test farm.

The installation succeeded. However, AutoSPInstaller configuration script threw out error message "Install-SPService : The specified value for the LocStringId parameter is outside the bounds of this enum."

So I deleted the relevant databases created by AutoSPInstaller, then ran "SharePoint 2010 Products Configuration Wizard" to initialize the farm. Similar error showed up in ULS log: "The specified value for the LocStringId parameter is outside the bounds of this enum."

After some investigation, I gave it up. There are a few people got the same problem, but no solution. So I uninstalled SharePoint 2010 with SP2 from the server, and then installed SharePoint 2010 RTM, then SP2, then CU201412.

It worked very well.

The problem is fixed.

Just curious: Is there any one knows what caused this problem? Any one use "SharePoint 2010 with SP2 installation media" to install SharePoint successfully?

Monday, December 8, 2014

"AppFabric Caching Service" stopped working and the solution

I'd like to configure one SharePoint server to provide dedicated "Distributed Caching Service". However, during the test, the "AppFabric Caching Service" on all SharePoint servers stopped working. The error message is pasted at the end of this post, and it didn't help much.

From "windows services" management console, I noticed that "AppFabric Caching Service" was not running. Everytime I tried to start it, it stopped in a few seconds, and the same error message appeared in ULS and Windows Events log.

After lengthy trouble shooting (include re-installation of AppFabric), in the end, I think I found the solution.

The error message in Windows Events mentioned "UriFormatException", that is a good sign, which tells us that something wrong with the URL "net.tcp://spServer1:22233"

In "C:\Program Files\AppFabric 1.1 for Windows Server\DistributedCacheService.exe.config", in [configuration]->[dataCache]->[hosts]->[host], there is an attribute "name". This is the one which is used in the URL. By default, the server name is "localhost". "localhost" is supposed to be replaced by the real server name automatically. Unfortunately, in my case, the conversion didn't work well. So, I had to replace it with the real server name manually, and then remove the IP address definition in local windows hosts file, which is added by AutoSPInstaller

After that, I rebooted the SharePoint server, and "AppFabric Caching Service" works again.


========= Windows Events Log =========

Application: DistributedCacheService.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.UriFormatException
Stack:
   at Microsoft.ApplicationServer.Caching.VelocityWindowsService.StartServiceCallback(System.Object)
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()


Faulting application name: DistributedCacheService.exe, version: 1.0.4632.0, time stamp: 0x4eafeccf
Faulting module name: KERNELBASE.dll, version: 6.1.7601.18409, time stamp: 0x5315a05a
Exception code: 0xe0434352
Fault offset: 0x000000000000940d
Faulting process id: 0x20c0
Faulting application start time: 0x01d012913ac4d3e5
Faulting application path: C:\Program Files\AppFabric 1.1 for Windows Server\DistributedCacheService.exe
Faulting module path: C:\Windows\system32\KERNELBASE.dll
Report Id: 7a999bf3-7e84-11e4-8843-005056b40054

========= ULS Log =========

12/08/2014 10:02:40.06 w3wp.exe (0x1A1C)                       0x26D4 SharePoint Foundation         DistributedCache               ah24w Unexpected Unexpected Exception in SPDistributedCachePointerWrapper::InitializeDataCacheFactory for usage 'DistributedLogonTokenCache' - Exception 'Microsoft.ApplicationServer.Caching.DataCacheException: ErrorCode:SubStatus:There is a temporary failure. Please retry later. (One or more specified cache servers are unavailable, which could be caused by busy network or servers. For on-premises cache clusters, also verify the following conditions. Ensure that security permission has been granted for this client account, and check that the AppFabric Caching Service is allowed through the firewall on all cache hosts. Also the MaxBufferSize on the server must be greater than or equal to the serialized object size sent from the client.). Additional Information : The client was trying to co... a492d39c-784a-40a4-b74c-34bf8b195e0d
12/08/2014 10:02:40.06* w3wp.exe (0x1A1C)                       0x26D4 SharePoint Foundation         DistributedCache               ah24w Unexpected ...mmunicate with the server : net.tcp://spServer1:22233     at Microsoft.ApplicationServer.Caching.DataCache.ThrowException(ResponseBody respBody, RequestBody reqBody)     at Microsoft.ApplicationServer.Caching.DataCacheFactory.GetCacheProperties(RequestBody request, IClientChannel channel)     at Microsoft.ApplicationServer.Caching.DataCacheFactory.GetCache(String cacheName)     at Microsoft.SharePoint.DistributedCaching.SPDistributedCachePointerWrapper.InitializeDataCacheFactory()'. a492d39c-784a-40a4-b74c-34bf8b195e0d
12/08/2014 10:02:40.06 w3wp.exe (0x1A1C)                       0x26D4 SharePoint Foundation         DistributedCache               air4g Monitorable Token Cache: Failed to initialize SPDistributedSecurityTokenCache  Exception: 'Microsoft.ApplicationServer.Caching.DataCacheException: ErrorCode:SubStatus:There is a temporary failure. Please retry later. (One or more specified cache servers are unavailable, which could be caused by busy network or servers. For on-premises cache clusters, also verify the following conditions. Ensure that security permission has been granted for this client account, and check that the AppFabric Caching Service is allowed through the firewall on all cache hosts. Also the MaxBufferSize on the server must be greater than or equal to the serialized object size sent from the client.). Additional Information : The client was trying to communicate with the server : net.tcp://spServer1:22233     at... a492d39c-784a-40a4-b74c-34bf8b195e0d
12/08/2014 10:02:40.06* w3wp.exe (0x1A1C)                       0x26D4 SharePoint Foundation         DistributedCache               air4g Monitorable ... Microsoft.ApplicationServer.Caching.DataCache.ThrowException(ResponseBody respBody, RequestBody reqBody)     at Microsoft.ApplicationServer.Caching.DataCacheFactory.GetCacheProperties(RequestBody request, IClientChannel channel)     at Microsoft.ApplicationServer.Caching.DataCacheFactory.GetCache(String cacheName)     at Microsoft.SharePoint.DistributedCaching.SPDistributedCachePointerWrapper.InitializeDataCacheFactory()     at Microsoft.SharePoint.DistributedCaching.SPDistributedCache..ctor(String name, TimeSpan timeToLive, SPDistributedCacheContainerType containerType, Boolean encryptData)     at Microsoft.SharePoint.IdentityModel.SPDistributedSecurityTokenCache..ctor(String name, TimeSpan timeToLive, SPDistributedCacheContainerType containerType, Boolean encrptyData, TimeSpan minimumTo... a492d39c-784a-40a4-b74c-34bf8b195e0d
12/08/2014 10:02:40.06* w3wp.exe (0x1A1C)                       0x26D4 SharePoint Foundation         DistributedCache               air4g Monitorable ...kenExpirationWindow)     at Microsoft.SharePoint.IdentityModel.SPDistributedSecurityTokenCacheInitializer.Init(Object state)'. a492d39c-784a-40a4-b74c-34bf8b195e0d

Tuesday, November 11, 2014

How to dispose SPWeb and SPSite objects in PowerShell

Whenever we get the message below from ULS log, we need to monitor it carefully. If the highlighted number("16") keeps growing and rarely goes down, then it's pretty clear that there is some SPWeb or SPSite objects not been disposed properly.

Potentially excessive number of SPRequest objects (16) currently unreleased on thread 10. Ensure that this object or its parent (such as an SPWeb or SPSite) is being properly disposed. This object is holding on to a separate native heap.This object will not be automatically disposed.

Some objects, such as SPWeb.ParentWeb and SPSite.RootWeb don't need to be disposed manually. But in loop statement such as "foreach($site in $allSites)" or "foreach($web in $allWebs)", we have to call "Dispose()".

Another case is the objects created by "Get-SPSite" or "Get-SPWeb". I have to say it's quite easy (for me) to forget releasing them. Thanks for the post from Dave Wyatt, we get the "Using" feature just like what we have in C#.

Below is how I implemented it, and happy to share it here.

function UsingObject
{
    [CmdletBinding()]
    param (
        [Parameter(Mandatory = $true)]
        [AllowEmptyString()]
        [AllowEmptyCollection()]
        [AllowNull()]
        [Object]
        $InputObject,

        [Parameter(Mandatory = $true)]
        [scriptblock]
        $ScriptBlock
    )

    try
    {
        . $ScriptBlock
    }
    finally
    {
        if ($null -ne $InputObject -and $InputObject -is [System.IDisposable])
        {
            $InputObject.Dispose()
        }
    }
}

function DisposeSPSite([REF]$site)
{
    if ($site.Value -ne $null)
    {
        $site.Value.Dispose()
    }
}

function DisposeSPWeb([REF]$web)
{
    if ($web.Value -ne $null)
    {
        $web.Value.Dispose()
    }
}

UsingObject ($site = Get-SPSite -Identity $siteURL) {
$objSPWebApplication = $site.WebApplication
$siteCollections = $objSPWebApplication.Sites

foreach($itemSPSite in $siteCollections)
{
if ($itemSPSite.Url.IndexOf("Office_Viewing_Service_Cache") -ge 0)
{
DisposeSPSite ([REF]$itemSPSite)
continue
}

Blah, blah....

DisposeSPSite ([REF]$itemSPSite)
}
}

We still get the message below in ULS;

An SPRequest object was reclaimed by the garbage collector instead of being explicitly freed.  To avoid wasting system resources, dispose of this object or its parent (such as an SPSite or SPWeb) as soon as you are done using it.

It seems that after calling "Dispose()", the SPRequest resource is not released immediately. They are done by .Net garbage collection soon after.

One more question: Can we only use "Start-SPAssignment" and "Stop-SPAssignment" to release the resources? For small farm, maybe all right. But, it's definitely not recommended. With more and more resources occupied by one PowerShell session, the farm will slow down eventually, and it also generates huge ULS log files.


Any comments welcome!


References:

http://msdn.microsoft.com/en-us/library/aa973248(v=office.12).aspx#sharepointobjmodel__findingundisposedobjects

http://msdn.microsoft.com/en-us/library/microsoft.sharepoint.administration.eventseverity(v=office.15).aspx

http://davewyatt.wordpress.com/2014/04/11/using-object-powershell-version-of-cs-using-statement/

Wednesday, October 1, 2014

How to stop duplicate entries based on two or more columns in a SharePoint list?

Well, there is no OOTB support of that.

Quick search leads to two solutions: coding (event receiver) or InfoPath rule.

The cost of "coding" is high. We need developer to build it, need someone to manage the source code, and need to upgrade the assembly when upgrading SharePoint. And, the deployment will cause farm outage!

InfoPath solution also has its own problem: it is always with expensive "license" fee.

So, is there simple approach to resolve the problem?  Below is how I did it, with a simple declarative workflow.

The workflow doesn't prevent "duplicate entries", but can generate warning message, so users can fix the invalid data after the remind. That's good enough in most of the cases.


  • The structure of the test list (CombinedKey = Title & ProjectCode)



  •  Declarative workflow triggered by "new item" and "edit item" event


  •  Test result

Any comments are welcome!

Friday, August 29, 2014

The substitute of farm solutions

We all know SharePoint 2013 provides "Apps mode". Microsoft encourages all developers to build "Apps" instead of classic solutions. However, there are many jobs Apps can't do. (Andrew Connell has some brilliant thoughts about that.)

In my opinion, there is one tool been seriously underestimated: PowerShell.

PowerShell helps administrators to automate jobs. It helps developers to manage SharePoint environment. Besides that, it can replace all routinely triggered customized workflows and timer jobs!

I didn't build any SharePoint Timer Job for production (but have two open source projects for fun: Simple Reminder and Workflow Timer). Comparing to a scheduled windows task (see the screenshot below), it's not easy. Really.


How about workflow or workflow activity/action? They also need SharePoint developer to build it. I can build them, but unless the business requirement is extremely complex, I prefer to build something even normal administrators can maintain and do simple modification.

In my experience, most of the business requirements are small, simple but urgent. Besides, non-software company don't want to manage the source code of hundreds of visual studio projects!

PowerShell script doesn't include any unnecessary xml or definition file. It simply focuses on business logic, which make it short and easy understand.

One of the complain about PowerShell is about the supporting tools, such as Visual Studio and ISE. There is no full intellisense during editing at the moment(Auguest 2014). Comparing to C#, the debugging is hard.

However, in my opinion, the critical issue is the lack of client side API. Like "Apps", we don't want to run PS scripts on SharePoint server. Is it possible to run the script on an application server with no SharePoint installed? Currently, CSOM and Restful API are supported, but, comparing to server object model, there are too much restriction. Actually we can only do basic read/write (CRUD) operations at client side.

That's far from enough. Any thing we can do through "server object model", we need to be able to do through "client object model".

This sounds like a balance between "security" and "functionality", and Microsoft needs to find a way to meet the ends.

[update, 2014-10-09]

To configure Task Scheduler in Failover Cluster, we can follow this one for Windows Server 2012. Only Windows Server 2012+ support "Clustered Tasks".

Tuesday, July 29, 2014

What is SPListItem/SPFile anyway?

When I started to touch SharePoint years ago, it's natural to treat "List" as "SQL table". They have so many similar features:

Features in commonSQL ServerSharePoint
ContainerDatabaseSite
OperationsInsert,Update,Delete,QueryInsert,Update,Delete,Query
MetadataColumnsColumns
Data volumeBillions of recordsMillions of records
Event triggerYesYes
Query languageSQLCAML

Apart from the exciting SharePoint features, such as versioning, soon I noticed two differences.
  1. All SPListItems are stored in one SQL table;
  2. Compare to SQL Server, "Insert,Update,Delete" operations of SPListItem are extremely slow;
Now, can we treat a SPListItem as a record in SQL table, with advanced features and poor performance on modification? And, further more, can we treat SPList as a SQL table which allow site owners to customize?

Definitely NOT!

This is the conclusion I got after seven years of development and administration on SharePoint platform. Believe it or not, I feel that SPListItem/SPFile is actually Message, and SPList/SPContentType is Queue.

Let me show an example here.

If there are millions of items in a SPList, and we need to update the "Title" field of all data based on some rules, how should we do it? We cannot update data in batch, instead, we have to do the update one by one. This is how we process messages, not records.

Once we understand this point, then it's easy to understand why all SPListItems are stored in one SQL table, why the modification is so slow, and how we can utilize SharePoint in better way.

What do you think? Any comments are welcome!