Tuesday, February 7, 2017

Backup and restore SharePoint 2013 site collection across different CU versions

We have a project that vendor provided a SharePoint solution in their environment, we would need to back up the site and restore to our environment. We identified that their SharePoint version is different than ours. Here is the quick way we could restore it to different version of SharePoint.
  • Download HXD editor
  • Open the backup file
  • Locate the version number and change it
  • Save the backup


Now you could restore it to different SharePoint version.

If you restore the backup to higher SharePoint version, you could also attache the database, mount the site, and upgrade it.

Monday, February 6, 2017

Tips to managing SharePoint SSRS Reports with Powershell

When you migrate one site collection with SharePoint SSRS Report solution from one environment to another, you will need to update the data source link to reflect the updated data source site URL. It’s extremely time consume to update the .rdl data source link though UI since out site has over 600 .rdl files.  After struggled few days, we are able to utilize the Powershell to automate the process updating the SSRS data source link based on this blog and instruction from Léon Bouquiet. The Powershell snippet is listed below.

$reportserver = "spsbxserver";
$reportServerUri = "http://$($reportserver)/_vti_bin/ReportServer/ReportService2010.asmx?wsdl";
$newDataSourcePath = "http://$($reportserver)/sites/ePlayBook1/SSRS/SPDS.rsds";
$reportFolderPath = "http://$($reportserver)/sites/ePlayBook1/";
#$newDataSourceName = "SPDS"; # No need to change data source name in our case


$ssrs = New-WebServiceProxy -uri $reportServerUri -UseDefaultCredential
$ssrs.Timeout=200000  # Set timeout in case large entries return to cause timeout

$reports = $ssrs.ListChildren($reportFolderPath, $true)
$reports | Where-Object {$_.TypeName -eq "Report"} | ForEach-Object {

       $reportPath = $_.path
       Write-Host "Report: " $reportPath 
       $dataSources = $ssrs.GetItemDataSources($reportPath)
       $dataSources | ForEach-Object {

        Write-Host "Report's existing DataSource Reference ($($_.Name)): $($_.Item.Reference)";

             $proxyNamespace = $_.GetType().Namespace
             $myDataSource = New-Object ("$proxyNamespace.DataSource")         
             $myDataSource.Item = New-Object ("$proxyNamespace.DataSourceReference")
             $myDataSource.Item.Reference = $newDataSourcePath
             #$myDataSource.Name = $newDataSourceName  # No need to change data source name
 
             $_.item = $myDataSource.Item 
             $ssrs.SetItemDataSources($reportPath, $_)
 
             Write-Host "Report's DataSource Reference ($($_.Name)): $($_.Item.Reference)";
       }

       Write-Host "-----------------------------------------------------"
}


We had few issues when we come up the final solution to manage the SSRS Reports with Powershell. Here are few tips that will help you to avoid the issue we encountered.

1. The first issue you might have is the exception for SSRS "Object reference not set to an instance of an object. ".  

The complete ULS log like this below.

01/19/2017 14:19:17.62         w3wp.exe (0x261C) 0x35CC    SQL Server Reporting Services               Report Server Catalog            00000                Unexpected             Throwing Microsoft.ReportingServices.Diagnostics.Utilities.SharePointException: , Microsoft.ReportingServices.Diagnostics.Utilities.SharePointException: Report Server has encountered a SharePoint error. ---> System.NullReferenceException: Object reference not set to an instance of an object.     at Microsoft.ReportingServices.SharePoint.Objects.RSSPImpUser.get_LoginName()     at Microsoft.ReportingServices.SharePoint.Utilities.CatalogItemUtilities.CreateCatalogItem(RSSPFile file)     at Microsoft.ReportingServices.SharePoint.Server.Utility.GetSPFileProperties(RSSPFile file)     at Microsoft.ReportingServices.SharePoint.Server.SharePointDBInterface.InternalFindObjects(ExternalItemPath wssUrl, CatalogItemList& children)     at Microsoft.ReportingServices.SharePoint.Server.SharePointDBInterface.FindObjectsRecursive(ExternalItemPath wssUrl, CatalogItemList& childList, Security secMgr, IPathTranslator pathTranslator, Boolean appendMyReports)     --- End of inner exception stack trace ---;         abafcc9d-e8f3-00e4-e462-39437ade3034

This issue is normally cause by the miss match SSRS add-in on SharePoint server and old SSRS add-in version as described in this KB article.

The solution is to upgrade the SSRS add-in on SharePoint server. In our case with SQL database version as 11.0.6567.0, we have to upgrade SSRS add-in to from version 11.0.3000.0 to SSRS 2012 SP3 with version 11.0.6020.0. Please refer my precious blog on the detailed procedure to upgrade.


2. The second issue you might have is the timeout issue. The error looks like this below.

Exception calling "ListChildren" with "2" argument(s): "The operation has timed out"

The solution is to increase the SSRS proxy timeout. Here is example we added in the Powershell as below.

$ssrs.Timeout=200000 


3. The third issue is the Powershell performance is extremely slow since most SSRS Powershell examples will loop through the whole SharePoint farm sites and lists recursively. You can imaging the time to go through all lists of the far. The Powershell script is like this below.

$reports = $ssrs.ListChildren("/", $true)

The solution is to pass the site collection URL to the ListChildren function. As a result, it will loop though only that one site collection. Here is the updated Powershell script.

$reports = $ssrs.ListChildren("http://mysharepointserver/sites/ePlayBook1/", $true)

If you only need to update the SSRS report on the top level site not recursively, you can also pass the send parameter as $false. There are few other blog you could refer on this topic.

New you could enjoy migrating the site collection from environment to another without have to change all the data source links manually.

Friday, February 3, 2017

SharePoint server reporting service configuration menu missing for report definition files .rdl files

When you started to utilize SQL Server Reporting Services for SharePoint, you could upload report definition file (.rdl) configure it.  Normally you will see the configuration like this below.




However, we run into issues few times the configuration menu missing for .rdl files. After debugging through the issues, we identified few tricks that could resolve this issue. Here are the steps you might check in order to resolve this issue.

1. First you need to verify whether the report definition files uploaded to SharePoint still have extension as “.rdl”.

When you use few IDEs to download the report definition file (.rdl) and modified it, it might change the file extension to xml since they are indeed xml format. The file with .xml extension uploaded to SharePoint will not be recognized as report definition file. As a  result, the reporting service configuration menu will not be displayed.

2. Second you need to verify if Reporting Services Site Collection Features are enabled.

One of the three key features is Report Server Integration Feature site collection feature. If the feature is not enabled, you will not be able to use the SSRS features. As site collection admin, you can verify the feature from setting setting. If you are not site collection admin, you can verify this by adding a content type to your library. You should see the "SQL Server Reporting Services Content Types".




3. The third step you need to verify if the correct version of Reporting Services Add-in for Microsoft SharePoint installed correctly. You would like to install the same version of add-in as SQL server.

The latest version now in 2017 is Microsoft SQL Server 2012 SP3. There is issue for previous SSRS as mentioned in Microsoft blog. You can verify the SSRS version installed under installed programs.



You need to verify if the running service is also in the same version by open the reporting service log file under \15\WebServices\LogFiles directory. The version should look like this.

<Product>Microsoft SQL Server Reporting Services Version 11.0.6020.0</Product>

If the version ins the log is different as installed software, you will need to restart the reporting service.

If you need to upgrade your Reporting Services Add-in for Microsoft SharePoint, you will need to download the installer, uninstall, and then install again. If you run into issue that the installation is always rolling back with the following error. You need to make two changes before reinstall.

  • Make all web.config writable including web.config under reporting service web service
  • Remove all comments like below as described in MSDN blog. This sounds ridiculous but I mean remove ALL comments!

     <!--<remove name="Session" />-->



4. The forth step you need to verify reporting service is running successfully on the server.

Verify if Services on Server is running on SharePoint serer. Check SharePoint central admin and service.


You can view the reporting service web service from IIS and browse the service form browser. The URL for below server is like.



You can also browse the reporting service wdsl for the site you will work on as the following url.


You might need to restart the service in order to bring the service back online.

5. If you still could not resolve the issues by the step #1-4, the nest step is to reinstall or upgrade the SSRS and add-in. 

You should also find the version that is same version as your SQL database. The current version we used is SSRS 2012 SP3. During the installation, you need to select the add-in component.

The reporting service for SharePoint is not very complicated. However, there are limited documentation and examples. Hope this instruction will help.




Tuesday, January 31, 2017

Utilize Visual Studio 2015 with GitHub extension to improve development efficiency

After our source repository migrated to GitHub, SharePoint developers started to use either Git window commends or SourceTree to manage the source code while developing projects inside Visual studio.

The major issue for Visual studio will not be able to identify which branch you are and difficult to switch branches. Developers had hard time to jump around different tools during the development phase and merge release phase. Since Visual studio 2015 has much better GitHub extension, my goal is to education all developers to utilize the Visual studio 2015 with GitHub extension to manage the day to day development for GitHub projects. We will cover the following topics.
  • Install GitHub extension
  • Create local project and add to location GitHub
  • Create Github new project and publish local project to remote GitHub
  • Check whcih branch you are working on
  • Create local branch and sync to remote Github
  • Verify local changes and push local change code to remote
  • Tog between branches
  • Merge branches
  • Clone project and different branches to local
  • Pull changes from remote GitHub to local
  • Checkout from remote GitHub branch

1. Install GitHub extension.
If you have Visual Studio 2015 already installed, you could install it two different ways.
  • Install from GitHub extension for Visual Studio and install separately
  • Click Visual Studio 2015 Tools -> Extensions and Updates. Search Github and install “Github Extension for  Visual Studio”

If you have not install Visual Studio 2015, you can select Visual Studio custom installation and select GitHub component as in this blog.

2. Create local project and add to local GitHub
Create a SharePoint Ad-in SharePoint hosted demo project named “SharePointGitHubDemo”. Right click the project and click “Add Solution to  Source Control …”
You will find the files inside Visual Studio will have an locker icon on and local .git folder created to complete the local GitHub configuration.

In Team Explore view, you can also see local Git repository project listed.





3. Create Github new project and publish local project to remote GitHub
In Team Explore view, Click “Connect…” and then “GitHub Enterprise”. Under GitHub user name, password, and the GitHub Enterprise server url such as  “https://github.mycompany.com/”. You will connect to the remote Github.





Make some changes to some files and right click the "commit" from project explore on the file. You will be redirect to change page and “Commit All and Push” will push all changes to Github if you have the remote Github project created in advance. Otherwise, click “Sync” and then “Publish to GitHub”. Fill in the project name, description, and the organize name of the project. Click “Publish” button. New GitHub project will be created and code will be pushed.

4. Check which branch you are working on.
In Team Explore view, click “Home” icon and then “Branches”. The branch you are on is the one highlighted. The example below shows you are working on "develop" branch.



5. Create local branch and sync to remote Github
In Team Explore view, click “Home” icon and then “Branches”. All the branches will be displayed. Right click the branch you would  like to branch out like “master” branch and select “New local branch from”. Enter the name of the branch and create. The new local like “newBranch” branch will be created.

Update some files and then click “Sync”. Then the “Push” under “Outgoing Commits”. You may need to click “Publish” if this is the first time to commit to remove. The develop branch changes from local will be pushed to remote “newBranch” branch.


After the sync, the icon on the file will changed to "Checked in".

7. Tog between branches
In Team Explore view, click “Home” icon and then “Branches”. Double click the branch you like to go. The branch will be highlighted.

8. Merge branches
In Team Explore view, click “Home” icon and then “Branches”. Click the "merge" link. You can select the branch you like to merge into the current branch. In this example, we will merge "develop4" branch into "master".



9. Clone remote GitHub project and different branches to local
In Team Explore view, click “Home” icon and then “Manage Connections” icon. Click "Clone" under "Local Git Repositories". Enter the Github project URL and local GitHub project location. Click "clone" and you will have the project cloned in local.


The default branch on the local will be master branch. You will need to right click project and "New branch from..." and then select the branch from GitHub you like to create to your local. You can pull all different branches you are interested in to your local.

10. Pull changes from remote GitHub to local
After you clone the local branches, you might want to make sure you have the latest code. You can do this in Team Explore view, click “Home” icon and then "Sync". Click "Pull" to pull all changes from Github to local. 

Click "Home" icon and double click the project at the bottom. The project will be open in "Solution" explore from local.

11. Checkout file change history
In "Solution Explore" view, right click the file and then "View history". You can see the change history.



At this point, you should be able to use Visual Studio for your development and integrate with GitHub without leave Visual Studio. We can also use Visual Studio to ingrate with "Team Services' for CI and CD.

One action from GitHub that is normally we use at end of the development phase is to add release tag. I'm not able to find how to do it directly from GitHub extension at this time. You can find other Visual Studio 2015 update 3 new features here.


Monday, January 16, 2017

SharePoint documents under folders with names forbidden on window like "AUX" are not searchable

When we migrated documents from file share to SharePoint 2013 document libraries, users reported that some files are not searchable. When we looked at the search craw logs, we identified many “The object was not found.” Errors. After looking closely at these documents, we have noticed that they are all under folder named “AUX”. In another word, any document under folder named “AUX” will fail in the crawl and will not be searchable.




AUX is one of the folders that is forbidden on window and there are many other folders similar to it. We believe SharePoint may not be able to craw any documents under all of such window forbidden folders. However, we are not able to find any information from SharePoint side.

If you find documents are not searchable, you might need to verify if the documents are under such folder that is forbidden on window.

As of May 6, 2017, we have received feedback from Microsoft that Microsoft Product group has triaged this issue and decided not to fix this issue at this time but they have added this to product backlog (will be fixed later). The reason for not fixing now is that the fix is very complicated and could touch code base that’s sensitive to changes and will require a lot of testing.

But…They have modified MSDN documentation to list out restricted filenames that could cause this issue…

We will have to follow up with Microsoft in the future.

Friday, December 2, 2016

Auditing solutions deployed to SharePoint environments

With increasing SharePoint 2013 customizations with new app model, there are incidents that these customizations are not working correctly in production while they are working on non-production environments. As SharePoint architect, I’m working with the team to come up an deployment auditing process to ensure what we deployed is what we should deployed. This auditing process will reports the details of the customization components especially the version labels that has been beneficial for deployment verification and debugging. Here are some the auditing reports that have helped to eliminated any customization environment discrepancies.

1. Audit app deployed to certain site collection and the app details including the versions. Here is the powershell commands.

param([string]$siteUrl)

$snapin = Get-PSSnapin | Where-Object {$_.Name -eq 'Microsoft.SharePoint.Powershell'}
    if ($snapin -eq $null) {
      Add-PsSnapin Microsoft.SharePoint.PowerShell -ErrorAction SilentlyContinue
}

############################################################################################################################################
function ListApps([string] $siteUrl, [string]$reportName)
{
        $appInstances = Get-SPappinstance -web $siteUrl

        if($appInstances -ne $null)
        { 
                foreach ($eachAppInstance in $appInstances)
                {        
                    $title = $eachAppInstance.Title
                    $version = $eachAppInstance.App.VersionString
                    $AppPrincipalId = $eachAppInstance.AppPrincipalId
                    $Id = $eachAppInstance.Id

                    Write-Output "$title; $version; $Id; $AppPrincipalId"| Out-File $reportName -append 
    
                }
        }


}

############################################################################################################################################

ListApps https://sharepoint.mycompany.com/sites/criticalsite E:\Applist.txt


The result is like below and you could comapre the result across different environments. There are references you could use for different app API.



2. Audit any customized dlls and the versions. Here are some powershell commands.

This one will list ALL dlls inside the current directory recursively.

Get-ChildItem -Filter *.dll -Recurse | Select-Object -ExpandProperty VersionInfo


This one will list only the named dll.

Get-ChildItem -Filter *.dll -Recurse | Where-Object {$_.name -Match "mycompany.com.sp.application1.common.dll"} | Select-Object -ExpandProperty VersionInfo 



3. Audit all farm solutions and their status. Here are the powershell commands.

Add-PSSnapin Microsoft.SharePoint.PowerShell -ErrorAction SilentlyContinue

# Get all Solutions deployed
Get-spsolution | select * |out-file c:\solution.txt

# Connect to the Farm
$SPfarm = [Microsoft.SharePoint.Administration.SPFarm]::get_Local()

# What Solution are we looking for?
$solution = "mycompany.com.sp.apps.workflows.wsp";

# Get the solutions
$currentSolution = $SPfarm.get_Solutions() | Where-Object { $_.DisplayName -eq $solution; }
$currentSolution


We are adding other auditing such as features, permission, configurations at this time and will share those in the future.

Friday, March 4, 2016

How to generate pfx certificate using cer certificate?

When we request SharePoint server to server trust certificates from the company, we are receiving the following two files.

Mycert.cer
Mycert.key

However, we do not receive the pfx file that required for SharePoint servers. We has been using different ways to construct the pfx file and I would like to summarize here in order for me to refer in the future.

There are different situations that we need to generate pfx certificate. The way to generate it will depends and I’ll show two different ways.

The first situation is  you have a private key with a .p7b certificate file and need to create a .pfx file.
You could use IIS server MMC UI to create the pfx file as described here.

The second is you have received both cer and kay file, you have to generate the pfx file from scratch. You could use the following command to generate the pfx file. You MUST put the key file with same name as cer file in the same directory as described here.

certutil -MergePFX Mycert.cer Mycert.pfx

The third situation is you have pfx file already imported to IIS but accidentally deleted. Now you only have the cer file but NO key file. Here is the way to generate the pfx file.

Upload the cer file to IIS as described in Microsoft support blog. Run the following command to restore the pfx file.

certutil -repairstore my "SerialNumber"

SerialNumber is the serial number that you find for the cer file uploaded.


Now you have the cert file for SharePoint server to use.