Procedure, tips, and verification scripts to build a server-to server trust between SharePoint Server 2013 and SharePoint Online for one-way outbound hybrid search

After replacing the STS certificate in the on-premises SharePoint farm described in previous blog, you should be ready to set up server-to-server authentication for hybrid environments. You need to create a trust relationship between your on-premises SharePoint farm and your SharePoint Online tenant, which uses Azure Active Directory as a trusted token signing service. This process described in technet will be done on one of the SharePoint on-premises WFE server that has online service management tools installed. Since there are several errors in the technet, we had to cross check other two instructions. One is published by Microsoft escalation engineer MANAS BISWAS and  Bill Baer. Here are the detailed steps, tips, and tricks.

1. First, you need to install the following online service management tools on ONE on-premises SharePoint Server 2013 web server.

• Microsoft Online Services Sign-In Assistant for IT Professionals BETA (64 bit version)
• Azure Active Directory Module for Windows PowerShell (64 bit version)
• SharePoint Online Management Shell (64 bit version)

There are some tricks and tips.

• Only install the BETA (64 bit version) of Microsoft Online Services Sign-In Assistant for IT Professionals. If you installed versions like RTW, you will not be able to install Azure Active Directory Module for Windows PowerShell. You will get MOSSIA is not installed error. At this time, you MUST install Microsoft Online Services Sign-In Assistant for IT Professionals with BETA (64 bit version).

• After Microsoft Online Services Sign-In Assistant for IT Professionals installed, the registries will be updated and server bounce is required. If the install did not prompt you to restart the server, reinstall and do a repair. It will then ask you to bounce the server.

By adding the required Windows PowerShell modules and snap-ins, the following process can occur in a single Windows PowerShell window on the on-premises SharePoint web server. 

2. Second, you will execute the following powershell on on-premises SharePoint WFE to create the trust and proxy.

# Set environment
Add-PSSnapin Microsoft.SharePoint.PowerShell
Import-Module Microsoft.PowerShell.Utility
Import-Module MSOnline -force –verbose
Import-Module MSOnlineExtended -force –verbose
Import-Module Microsoft.Online.SharePoint.PowerShell -force

# Set the certificate
$stscertpfx="E:\source\O365\O365DEV.pfx"
$stscertcer="E:\source\O365\O365DEV.cer"
$stscertpassword="Qualcomm1"
$spcn="*.qualcomm.com"
$spsite="https://sharepointdev.qualcomm.com/"
$spoappid="00000003-0000-0ff1-ce00-000000000000"


# Update the Certificate on the STS - You could skip this if you already done this
$pfxCertificate=New-Object System.Security.Cryptography.X509Certificates.X509Certificate2 $stscertpfx, $stscertpassword, 20
Set-SPSecurityTokenServiceConfig -ImportSigningCertificate $pfxCertificate

# Type Yes when prompted with the following message.
#You are about to change the signing certificate for the Security Token Service. Changing the certificate to an invalid, inaccessible or non-existent certificate will cause your SharePoint installation to stop functioning. Refer to the following article for instructions on how to change this certificate: http://go.microsoft.com/fwlink/?LinkID=178475. Are you sure, you want to continue?

#Restart IIS so STS Picks up the New Certificate - need to be done on all SharePoint servers
iisreset
net stop SPTimerV4
net start SPTimerV4

#To validate that the above commands has run successfully, you can run any of the following cmdlets. The certs should be matching
$pfxCertificate
(Get-SPSecurityTokenServiceConfig).LocalLoginProvider.SigningCertificate


#Do Some Conversions With the Certificates to Base64
$pfxCertificate = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2 -ArgumentList $stscertpfx,$stscertpassword
$pfxCertificateBin = $pfxCertificate.GetRawCertData()
$cerCertificate = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2
$cerCertificate.Import($stscertcer)
$cerCertificateBin = $cerCertificate.GetRawCertData()
$credValue = [System.Convert]::ToBase64String($cerCertificateBin)

# Establish Remote Windows PowerShell Connection with Office 365
enable-psremoting

#When prompted with Are you sure you want to perform this action? type Yes for all of the actions.
new-pssession

# Log on as a Global Administrator for Office 365
Connect-MsolService

#When prompted, provide the Global Admin account for your Office 365 tenant. This would have been sent to your corporate e-mail address when you signed up for the tenant.

# Register the On-Premise STS as Service Principal in Office 365
New-MsolServicePrincipalCredential -AppPrincipalId $spoappid -Type asymmetric -Usage Verify -Value $credValue




# Add an SPN for your public domain name to Azure Active Directory
$SharePoint = Get-MsolServicePrincipal -AppPrincipalId $spoappid
$spns = $SharePoint.ServicePrincipalNames
$spns.Add("$spoappid/$spcn")
Set-MsolServicePrincipal -AppPrincipalId $spoappid -ServicePrincipalNames $spns
$spocontextID = (Get-MsolCompanyInformation).ObjectID
$spoappprincipalID = (Get-MsolServicePrincipal -ServicePrincipalName $spoappid).ObjectID
$sponameidentifier = "$spoappprincipalID@$spocontextID"

# Set the SharePoint authentication realm
$site=Get-Spsite "$spsite"
$appPrincipal = Register-SPAppPrincipal -site $site.rootweb -nameIdentifier $sponameidentifier -displayName "SharePoint Online"
Set-SPAuthenticationRealm -realm $spocontextID


# Configure an on-premises proxy for Azure Active Directory and establish in the On-Premise Farm a Trust with the ACS
New-SPAzureAccessControlServiceApplicationProxy -Name "ACS" -MetadataServiceEndpointUri "https://accounts.accesscontrol.windows.net/metadata/json/1/" -DefaultProxyGroup
New-SPTrustedSecurityTokenIssuer -MetadataEndpoint "https://accounts.accesscontrol.windows.net/metadata/json/1/" -IsTrustBroker -Name "ACS"



3. Third, you need to verify the O365 certs match the on-premises, proxy configured, and ready for hybrid search configuration.

A. Verify SharePoint server and STS service have the identical certificates. You could use the following powershell and verify the thumbprint.

# Verify whether the SahrePoint on-premises server certificate match the STS service certificate   
Add-PSSnapin *sh* -ea 0
$stscertpfx="E:\source\O365\O365Sbx.pfx"
$stscertcer="E:\source\O365\O365Sbx.cer"
$stscertpassword="Qualcomm1"
$spcn="*.qualcomm.com"
$spsite="https://sharepointdev.qualcomm.com/"
$spoappid="00000003-0000-0ff1-ce00-000000000000"

#Validated STS Token Signing certifciate thumbprint
$pfxCertificate=New-Object System.Security.Cryptography.X509Certificates.X509Certificate2 $stscertpfx, $stscertpassword, 20
$pfxCertificate

# This cert is from SharePoint STS service
Get-SPSecurityTokenServiceConfig).LocalLoginProvider.SigningCertificate 
 

B. Verify certificate uploaded to O365 not expired using the following commands.
Connect-MsolService
Get-MsolServicePrincipalCredential -AppPrincipalId "00000003-0000-0ff1-ce00-000000000000"

You need to press enter and see the output with expiration data for the cert.

C. Validate SPNs setup properly in O365 using the commands.
$app = Get-MsolServicePrincipal -AppPrincipalId "00000003-0000-0ff1-ce00-000000000000"
$app.ServicePrincipalNames

You can verify the domain for the SharePoint on-premises should be in the list.

PS C:\Users\SPdev1> $app = Get-MsolServicePrincipal -AppPrincipalId "00000003-0000-0ff1-ce00-000000000000"
$app.ServicePrincipalNames
00000003-0000-0ff1-ce00-000000000000/*.qualcomm.com
00000003-0000-0ff1-ce00-000000000000/*.sharepoint.com
00000003-0000-0ff1-ce00-000000000000
Microsoft.SharePoint

D. Validate User Profile Service Application Status and it should be "online"
$upa=Get-SPServiceApplication | where-object {$_.TypeName -match "User Profile Service Application"}
$upa
$upa.status

E. Validaye on premises ACS Proxy created and it should be the named you configured in previous script. In this case, it's "ACS".
Get-SPServiceApplicationProxy | ? {$_.Name -eq "ACS"} | fl

You could check SharePoint central administration->Manage Service Applications. You should see new ACS Azure Access Control Service Application Proxy listed.

F. You can also verify the trust from SharePoint central administration->Security->Manage trust. The ACS proxy should be created as in the following screenshot.

Next it is ready to continue the search hybrid configuration. Please note there are some errors in the Microsoft technet article and the procedure we verified is based on Microsoft escalation engineer Manas.

One tip is you could suppress the  users import like login name, password, and the confirmation when you run the above powershell commands. You could also encrypt the password in the file.

Here is the procedure to encrypt the password. Login as the user account to window server and run the following script and the password will be encripted in the file.

read-host -AsSecureString | ConvertFrom-SecureString | out-file C:\cred.txt

You could use this file to auto accept the password. Here is the automated commands that will not prompt any input. The highlighted parameters are the key for the automation.


$o365admin = "userID@domain.company.com"
$password = get-content C:\cred.txt | convertto-securestring
$Creds = New-Object System.Management.Automation.PSCredential($o365admin, $password)

enable-psremoting -Force

#When prompted with Are you sure you want to perform this action? type Yes for all of the actions.
new-pssession

# Log on as a Global Administrator for Office 365
Connect-MsolService -Credential $Creds

See Ultimate procedure to display SharePoint online hybrid search results in SharePoint Server 2013 for other steps to configure hybrid search. 

The mismatch functions/features between on premise SharePoint and Office 365 you need to review when implementing SharePoint hybrid mode

When you install Service Pack 1 for SharePoint Server 2013, you can redirect on premise users to OneDrive for Business in Office 365 when they click OneDrive or Sites in the navigation bar. This way, no matter where they are, they can quickly access their documents and any information that they choose to sync from their SharePoint sites without login to company's network. You could also configure the audience targeting to redirect only selected group of SharePoint users in hybrid mode while other users are still on premise.

The move to Office 365 hybrid mode has been a good thing. Even so, a few things caught me during the testing. So I wanted to pass along a list of mismatch functions/features to consider before subscribing to Office 365 for all SharePoint users. Some of the functions/features that have been removed or retired from Office 365 are as designed and I could see good reason behind. However, this might cause some issues if these functions are already used on premise and need to be migrated to Office 365. Here is the list of the mismatch functions/features.

1. The “Tasks” link in your MySite has been removed
2. SharePoint Tags & Notes has been retired
3. Available site template types for sub site creation will be limited
4. Available apps could be added will be limited
5. Available web parts to add to page will be limited
6. Global managed metadata term store might not be migrated
7. Other customizations especially server side customizations

Here are the details of the mismatch functions/features for your reference for either end user communication or content migration.

1. The “Tasks” link in your MySite has been removed as described in Microsoft support site - One of the new features in SharePoint 2013 is the ability to have an overview of the tasks assigned to a person. With SharePoint 2013, the source of the aggregation has diversified including now Project Server Tasks, SharePoint Tasks and Exchange Tasks , all these aggregated in the special "tasks" view in each user's my site. There is also a “Sync to Outlook”, so that all your tasks can be accessed within Outlook and be available anywhere, including your smart phone.

The following is a screenshot from on premise, which shows all the tasks that are currently due, or that you’ve marked as important. The tasks can be grouped by source (SharePoint sites, Project Server projects, personal list, or your Exchange mailbox), and can be viewed as a Gantt chart, which is more descriptive than a list of scattered due dates.

We noticed the Tasks link has been removed form MySite and added to the top link. The new Office 365 new Taks link will display the tasks in exchange. If we have not enabled 365 exchange, we might not be able to view aggregates tasks. In additional, the project server 365 is different license and the on premise project server tasks will not be displayed if we do not migrate to 365 project server.
 

 2. SharePoint Tags & Notes has been retired - Tags & Notes button on ribbon still visible but disabled from Office 365. The following screenshot is from SharePoint on premise and the Tags & Notes button is enabled.

The social tags will no longer appear in the tags refiner. . Some other related Tags & Notes feature retired are described in SharePoint online article. Here's what Microsoft recommend as an alternative to Tags & Notes:

• Yammer (see Say hello to Yammer)
• SharePoint newsfeed (see Pick your enterprise social network: Yammer or Newsfeed?)

3. Available site template types for sub site creation will be limited – You will have different site templates when creating sub site on premise and in office 365. The following screenshot is from SharePoint on premise when you creating sub site from MySite. You will see “Community Site” and "PowerPivot site" are available templates. Please note, your SharePoint administrator can configure to hide some site templates.

 

The “Community Site” site template has been removed from office 365 and you could not create new community site. It would be the similar reason that Microsoft promotes Yammer as an alternative. However, if you have community site created under MySite and you would need to migrate, you have to use migration tool to migrate the content to different site template.

The "PowerPivot site" has been removed from office 365 at this time. Microsoft has come up a new feature named “Power BI sites on Office 365” as an alternative. This new Power BI sites on Office 365 has great feature that have not been implemented on premise. The same concern is if you have PowerPivot site created under MySite and you would need to migrate, you have to use migration tool to migrate the content to different site template.

Office 365 introduced a new site template under “Duet Enterprise” named “SAP Workflow Site”. We have to played around this template but it seems to utilize an SAP Workflow site that aggregates all user business tasks. You can add other Web Parts related to that workflow. This might be a future alternative to the “Tasks” link in your MySite.

4. Available apps could be added will be limited from office 365. This could be changing based on Microsoft 365 releases.

As you can see in the following two screenshots, the apps you could add from Office 365 is different from on premise. This might also cause issues when you try to migrate the content from on premise to Office. If the content is from apps not “supported” on Office 365, you will have migration errors. This could be resolved by converting existing apps to other apps supported by Office 365 through third party migration tool. We might need to evaluate whether we need to deal with such issue.

Here is the screenshot for  apps you could add for Office 365. There is limited choices at this time.

Here is the screenshot for  apps you could add for on premise. There is much more choices at this time.

5. Available web parts to add to page will be limited. The is understand that Office 365 has different webpart that you could add to your page.

The following screenshot shows the webpart you could add from on premise.

 

The following screenshot shows the webpart you could add from Office 365. The webpart available from Office 365 is much less. Again, Microsoft is doing great job to enhance the Office 365 as much quicker pace than on premise. There might be more webpart available in the near future.

6. Global managed metadata term store might not be migrated – Managed metadata term is a taxonomy or formal classification system. A taxonomy groups the words, labels, and terms that describe something, and then arranges the groups into a hierarchy. The global managed metadata term store will provide the taxonomy for the whole farm. If some of the MySites are using the global term coming from on premise, it will be displayed blank when migrated to Office 365. We have not done additional research to verify whether there is need to migrate the terms with the content if there is any. This will not be an issue if global managed metadata terms are not used on MySites.

The following screenshot is for Office 365 managed metadata term store. We would need to do future research on the utilization of this managed metadata term store.

7. Other customizations especially server side customizations could not be implemented in Office 365. This is obvious that any customizations you have done on premise may not directly migrated to Offcie 365. Some of the functions provide either by third party or customization might not be migrated and need to be reviewed during the content migration.

The third party tools like Quick Apps for SharePoint this is widely used that might not be migrated. The customized master pages, feature stamping to enable the auditing and version auto enabling will be left behind. We might need to remove these customizations before migrating the content to Office 365.

There might be some other confusions and questions need to be figured out. There will be two separated ‘About me links for both on-premise and cloud. What link will other users (who are not part of Office 365 pilot) be directed to if they click on the People Picker field of an Office 365 user?

As a summary, Office 365 hybrid mode has been a promising and we would need to plan the smooth transition and content migration.

SharePoint 2013 issues on Internet Explorer 11

When I was doing labs during the SharePoint training, several SharePoint 2013 issues are identified accidentally against Internet Explorer 11. The most annoying one is that SharePoint pages are not in full edit mode when clicking edit page. One example is web part properties cannot be modified since the link will not displayed on the page as in the following screenshot.

 

Some other IE 11 comparability issues below also reported as this point.

• Calendar web part is extremely corrupted
• The calendar overlay button on the calendar view is disabled
• Edit Page doesn’t place the page in Edit mode (especially on custom page layouts)
• If you do happen to use a built-in page layout, any webparts added are unable to be customized
• Drag and drop files to document library seems to be disabled

There are at least two ways to resolve these issues as listed below.

1. Update IE 11 compatibility mode
2. Update master pages to IE10 Compatibility

The easy one would be the option #1 and here are the detailed steps.

• Place your Top Level Domain (ex. Contoso.com)  in Compatibility Mode  
• Place your Top Level Domain (ex. Contoso.com) in the Intranet Sites Security Zone

There might be some other issues we might need to find out.