Tips to change SharePoint 2010 behavior without change out of box UI - Part II SPCustomLayoutsPage

Modifying the files that are installed by SharePoint is not supported to change the behavior of the SharePoint. As we discuss before, one of the best practices is to use the tagmapping functionality to instruct the parser to substitute a different derived type implementation whenever it encounters the type being mapped. It is quite useful to redirect SharePoint Out of box to customized one. However, this mechanism will be difficult if you need to customize the behavior of out of box application pages such as AccessDenied, Confirmation, Error, Login, RequestAccess, Signout and WebDeleted. In this blog, we will introduce the new SharePoint 2010 Powershell Set-SPCustomLayoutsPage that will replace application pages in layouts directory that is supported by Microsoft.

Here is the user case for our company - users want to customize the access denied page to have the following behavior and we also prefer to apply the same company branding to the page.

Include the following sentence “You are requesting access to highly sensitive company content, which requires a business justification, and in some situations, approval from your supervisor(s). In order for your request to be properly and quickly assessed, please specify what project you are working on, the specific content in which you are interested, and (most importantly) why you need it.”

  

The solution provided by SharePoint 2010 is very simple and here are the steps. We use RequestAccess page as example.

1. Copy the access deny application page from layout folder and copy/modify to incorporate the change required including the branding

2. Deploy the custom Request Access page to custompages folder under layouts directory

3. Set custom page for SharePoint to use instead

Set-SPCustomLayoutsPage -Identity "RequestAccess" -RelativePath "/_layouts/custompages/reqacc.aspx" -WebApplication "{replace with web app url}"

Now the access denied page will use the custom one instead of the OoB one. You could verify what custom page SharePoint is using by typing the powershell Get-SPCustomLayoutsPage command.

You might also change it pro grammatically through IHttpModule or event receiver that might be fit into your use cases. If you need to have a different access denied page for different site collection, you might add logic to the custom page or createdifferent folder each site collection. 

In addition, if you need to display more information to the customized access denied page such as owner list, you could use server object model API.

Major SharePoint 2013 upgrade issues and workarounds

After looking at SharePoint 2013 upgrade path, it’s clear that in-place upgrade is never recommended since you might also need to use new Window 2012 server and SQL 2012 server. The recommended migration procedure and I would see the only procedure is database-attach upgrade approach.

Here is standard upgrade procedure from Microsoft “When you use the database-attach upgrade approach to upgrade from SharePoint 2010 Products to SharePoint 2013, you upgrade the content databases and several service application databases. You do not upgrade the configuration database for your farm.” With this approach, there are many issues after the upgrade and I’ve listed some of the major ones in this blog.  Some of the issues we have workarounds but others we do not have solution.

1. The first issue is email enabled lists will no longer receive emails. The upgrade procedure indicated “do not upgrade the configuration database for your farm” and the email enabled list configuration is inside configuration database. We have encountered this issue in 2007 to 2010 upgrade and there are many ways to resolve this.

The easiest solution as I published earlier is to copy EmailEnabledLists from 2010 configuration database to 2013 configuration database.

2. The second issue you might have is the “In Progress” workflow tasks created from Out of the Box workflows cannot be completed. This should be same issue published before after installing Service Pack 1 for SharePoint Server 2010 or upgrade from 2007 to SharePoint 2010. 

Every workflow task has a reference to its workflow definition. When there is an in-flight workflow and its tasks, the Workflow definition file of the original Out of the Box workflows is replaced with new ones if we upgrade the farm or Site Collection. This makes the tasks of the previous version of Out of the Box workflows lose the reference to their workflow definition and get into locked state when a task assigned attempts to update (complete like “Approve” or “Reject”). Eventually workflow instances ofthe previous version of Out of the Box workflows cannot be completed. 

Workaround is to complete all “In Progress” workflows before the upgrade or “terminate this workflow now” menu in workflow status page shown below, which will also remove all the locked tasks related to that workflow.

3. The third issue is the major and critical one that all managed metadata data columns un-editable and terms are not available even you associate the new MMS with the migrated database. The issue applies to not only global terms but also local terms. This is expected since we have this issue on 2010 when we migrate content from one farm to another.

Solution is to use toll we developed with Microsoft to fix managed metadata data as we discussed before. We might need to recompile the code against 2013.

4. The forth issue is SharePoint Web Analytics system will no longer be available or upgradable to SharePoint 2013 since web analytical service will be replaced by new 2013 search service. The previous analytics reports will not be imported into new version and there is no procedure from Microsoft how we migrate existing web analytical reports.

Solution is unknown and I’m following up with Microsoft on this.

5. The fifth issue is around removed templates and features. One example is the Group Work site template is no longer available as a site option. The issue is whether existing sites created from Group Work site template will be able to upgrade to 2013. We had issue fab40 sites are able to beupgraded to 2010 from 2007.

Solution is unknown and I’m following up with Microsoft on this.

There are many other issues you may encounter for 2013 upgrade and we will share our findings.

What architecture changes in SharePoint 15 (2013) and what you to prepare as a architect?

Microsoft announced the release of the public beta for SharePoint 2013 on July 16. There are many new features introduced with some architecture changes. A recent survey of attendees to BZ Media’s SPTechCon conference revealed that, of 166 respondents, 70.5% said they do not plan to adopt the next version of SharePoint for more than a year, while 20.5% said they will adopt within a year of general availability, and 9% responded they will adopt as soon as the software is generally available.

The summary from Microsoft on SharePoint 2013 is SharePoint Server 15 enhances the workload experiences by enabling new compelling scenarios that engage and work with the user. If you are the architect for your current SharePoint 2010 team, you should be aware of the change so you could be prepared to make the decision.

In general, SharePoint 15 (2013) model has stayed same as 2010 version. However, numerous platform level improvements and capabilities listed.

• Shredded Storage – Only update the changed content to storage
• SQL improvement – Support SQL 2012
• Cache Service – Distributed cache across servers to improve performance
• Request Management – Redirect request to individual server for large farm
• Themes – Better user interface
• Sharing – Improve content sharing and access control

The detailed major changed are summarized in the following categories. I will not copy all the content and please refer to Microsoft SharePoint 2013 resource center for details. The detailed video and presentation will help you to understand the details. Some summary also provided by some people.

1. Service application changes

• Office Web Apps is no longer as service application
• Web Analytics is no longer as service application, it’s part of search
• Other enhanced service applications

2. Enterprise Content Management

• Site –level retention policies
• Discover Center
• eDiscover capabilities
• Team folders to integrate with exchange

 3. Web Content Management

• Support the tools and workflow designers use
• Variations & Content Translation
• Search Engine Optimization
• Cross Site Publishing
• Video & Embedding
• Image renditions
• Clean Urls
• Metadata navigation

4. Social

• Microblogging – share and follow
• Activity Feeds – view activities related to content
• Communities
• Discussions
• Blogs 

5. Search

• Personalized search results based on search history
• Rich contextual reviews

6. Business Intelligence Enhancements

7. Mobile

8. Remove API Enhancements

When we looked at the architecture changes and major changes to us might be the office web apps and web analytics changes. There will be some design you should consider now before you run into the dead end. Here are the description, reason for change, and what you need to be prepared as architect for the two changes.

Description for change #1: Office Web Apps is no longer as service application. It is a separate application and recommended to be installed as separate farm.

Reason for change: Leverage Office Web Apps to integrate with SharePoint, exchange, Lync, and other third party application. The new architecture recommended is displayed in the screen shot.

 

What need to be prepared: Here are things you need to do as architect on SharePoint 2010.

If you have not deployed the Office Web Apps on SharePoint 2010 farm, serious consider NOT deploy this application unless this is absolutely needed by the business side. The critical issue for Office Web Apps installation on SharePoint 2010 is you could not un-deploy it unless to remove the server from the farm and rejoin as we discuss with Microsoft! The un-deploy will remove the servers from the farm and will take large amount time to rejoin all servers back with clean environment. Please see my previous blog for detail on the risks.

If you have Office Web Apps on SharePoint 2010 farm, you might consider removing it from the farm before the upgrade! This might be the simplest way since I’ve not heard any upgrade process.

Description for change #2: Web Analytics in SharePoint Server 2010 has been discontinued and is not available in SharePoint 2013 Preview. Analytics processing for SharePoint 2013 Preview is now a component of the Search service. Details in Microsoft site.

Reason for change: A new analytics system was required for SharePoint 2013 Preview that included improvements in scalability and performance, and that had an infrastructure that encompasses SharePoint Online. The Analytics Processing Component in SharePoint 2013 Preview runs analytics jobs to analyze content in the search index and user actions that are performed on SharePoint sites.

SharePoint 2013 Preview still logs every click in SharePoint sites and still provides a count of hits for every document. User data is made anonymous early in the logging process and the Analytics Processing Component is scalable to the service.

This analytics data is used in SharePoint 2013 Preview to provide new item-to-item recommendation features, to show view counts that are embedded in SharePoint 2013 Preview and Search Server user interface, to provide a report of the top items in a site and list, and to influence the relevancy algorithm of search.

What happens to Web Analytics after upgrade: The Web Analytics Service is not upgraded to the Analytics Processing Component in SharePoint 2013 Preview. When you upgrade to SharePoint 2013 Preview, the databases that contain the data from Web Analytics in SharePoint Server 2010 are not removed. These databases are not used by or maintained by the Analytics Processing Component in SharePoint 2013 Preview. This means that documents on sites in SharePoint Server 2010 that are upgraded will show a hit count of 0.

When you upgrade to SharePoint 2013 Preview, do not attach and upgrade the databases that contain the data from Web Analytics in SharePoint Server 2010. We recommend that you turn off Web Analytics in the SharePoint Server 2010 environment before you copy the content databases that you want to upgrade to SharePoint 2013 Preview.

Reports from Web Analytics for the top items in a site are carried forward. Reports that show browser traffic, top users of a site, and referring URL are not carried forward and are not used by the Analytics Processing Component in SharePoint 2013 Preview.

Administrative reports for the quota usage of site collections in the farm are not available in SharePoint 2013 Preview.

SharePoint 2013 Preview does not support the Web Analytics Web Part. After a farm is upgraded to SharePoint 2013 Preview, all instances of a Web Analytics Web Part will not function. The page that includes the Analytics Web Part will render and a message appears that informs the user that the Web Part is no longer supported.

 

What need to be prepared: You should generate a report what pages are using Web Analytics Web Part and remove them before the upgrade.You should design to utilize the new search application service and new BI functions to replace current Web Analytics functions.

SharePoint expert Chris McNulty from Quest will detail five specific actions to prepare for the future, including:

• Establish governance today
• Choose code-free customization
• Perform inventory and analysis
• Implement data externalization
• Consolidate content

There are many other architect and design you should implement now to be prepared for SharePoint 15 (2013).

What's new in SharePoint 15 (2013) and what you to prepare as a developer?

We are on good authority from multiple anonymous sources that the beta release of SharePoint 15 will drop next Monday July 16, 2012. It had been reported that Microsoft was looking for a summertime drop of the beta from the leaked road map, and it has hit the mark.

On July 11, 2012, Microsoft even has posted the details of a developer "FirstLook Clinic: What new in Sharepoint 15" course for SharePoint 15, which shows two types of hosted apps, the use of Agaves (HTML5 and JavaScript add-ons) to extend Office apps, and improvements in content management. The link was there for about two days and was removed on purpose by Microsoft now!

From the Microsoft developer “FirstLook Clinic: What new in Sharepoint 15” course, the executive summary of what we can learn are these features:

• SharePointmarketplace for Apps – New SPApp object on SPWeb level to to create an ecosystem for multitenant apps.
• Office Agaves, HTML add-ons for Office documents – Stronger integration with Office
• New enterprise services, including search from client-side object model - Enhanced enterprise services like managed metadata service 
• Workflow looping for Designer that is only available in Visual Stadio on 2010
• MinimaDownload Strategy
• OAuth2.0 Authentication
• Ability to integrate or platform for SharePoint education

From other resources including Microsoft class “10232A Design and Developing Microsoft SharePoint Server 2010 Applications” I toked this week and Microsoft “SharePoint 2010: 15 Technical Preview - Announcements and Resources” and and API published in February 2012, we might also expect the following major changes.

• .Net4.0 integration with SharePoint
• Workflow function 4.0
• HTML 5 with open source JQuery instead of Microsoft proprietary Ajax scripts
• ViewState is only supported on the legacy model due to culprit for not having testing and other things available
• Visual Studio 2012  

Most of the changes mentioned here will change our design and development decision. So we will discuss some of them in details.

The biggest advantage in .Net 4.0 is the MVC module. I heard SharePoint 15 will implement .Net 4.0 MVC to separate the module, controller, and view. This way, we have biggest advantage to utilize the HTML 5 and open source JQuery as view layer while utilize server controller components. As a result, you should try to avoid Microsoft proprietary Ajax script and separate your logic into web service or utility class.

In Workflow function 4.0, State machine workflows are deprecated in workflows foundation 4.0 runtime – replaced with a flowchart approach. As a result, you should not implement any State machine workflows at this time. If you are using third party state machine workflow from Nintex, K2, or AgilePoint, you might contact vendor.

HTML 5 has made its way to become standard and SharePoint 15 will leverage it with open source JQuery instead of Microsoft proprietary Ajax.  Try to avoid any Microsoft proprietary Ajax scripts.
  
Microsoft is giving up Silverlight component in the future reelases. As a result, if you are using Silverlight webparts, consider to replace any Sliverlight components.

ViewState is only supported on the legacy model in SharePoint 15 since it could not be unit tested. As a result, plan to disable the View state and use different method.

There are many new features on Visual Studio 2012. Some of them are significant than others. Here are some of them I’m looking for.

• Publish SharePoint solutions to remote SharePoint sites
• Increase SharePoint performance by using profiling tools
• Create sandboxed visual web parts
• Use Microsoft Fakes framework for Unit testing

You should plan to use Visual Studio 2012 and set up your unit tetsing code now and untilize future unit testing framework on .Net 4 framework.

With Microsoft $1.2 Billion Yammer Buy on June 25, we could expect much social functions in future SharePoint and be prepared to utilize social functions whence there are available. Window 2012 server and Window 8 technology will also impact SharePoint 15 development.

Some suggestions from other people to prepare for SharePoint 15 are also valued. We'll closely monitor the SharePoint 15 progress and post what we found.

For your information, Microsoft has put together some great resources to get you started. They are all in one convenient location - the SharePoint Server 2013 Resource Page. For example, check out:

·	SharePoint Server 2013 Deployment Guide
·	Microsoft Office Web Apps Server Preview
·	SharePoint Team Blog

Managed metadata terms are not visible on document library item edit property dialog after migrated from one site to another using powshell commands

As we mentioned before, the major issue of using managed metadata terms is that the managed metadata column values become un-editableand terms are not available after the content migration either fromdifferent farm or different webapp. We have some solution that will be published later to deal with most of the situation or use cases as list before. During the testing of the solution on use cases #5 and #6 as listed below, we found there is issue on the powershell backup/restore or export/import commands to migrate terms associated with SharePoint document library. The terms are not visible on document library item edit property dialog window!

• Migrate sub site(s) from one site collection to a different site collection with the same Managed Metadata Service (MMS)
• Migrate sub site(s) from one site collection to a different site collection with the different Managed Metadata Service (MMS)

In order to explain the details, we will create one sub site with one custom list to compare with document library. We will also use saved template to compare with export/import process.

Here are details steps to reproduce this issue and explain the issue.

• Create a team site http://sbx08:3797/sites/MSource and a sub team site http://sbx08:3797/sites/MSource/Sub1
• One sub1,  modify Shared Documents library by adding one column to use global term and another column to use local term as in the screen shot

• One sub1, add one custom list with one column to use global term and another column to use local term as in the screen shot

• One sub1, you will see you are able to see terms when edit items on both Shared Documents library and custom list

 

• Export the sub1 sub site (web)

        export-spweb http://sbx08:3797/sites/MSource/Sub1 -path c:\webpackage.cmp -CompressionSize 100000000 -verbose

• Create a destination site collection http://sbx08:3797/sites/MDestination and a sub team site http://sbx08:3797/sites/MDestination/SubD1
• Import the sub1 sub site (web) to destination site collection and see the screen shot for the sub site on destination

       import-spweb http://sbx08:3797/sites/MDestination/SubD1 -path c:\webpackage.cmp -force -verbose

• When you try to edit properties the item inside Shared Documents library, you will find the managed metadata columns disappeared and NOT visible!

This is same if you use backup/restore powershell commands. However, the managed metadata columns on list are visible! Of cause, the managed metadata are not linked correctly as we mentioned before that will be addressed in future blogs.

 

In order to provide stranger evidence on this powershell issue, we will use the following steps to show the expected behavior.

• Save the sub1 http://sbx08:3797/sites/MSource/Sub1 to a template WITH content named MTemp1.wsp
• Save MTemp1.wsp solution to local disk as MTemp1.wsp
• Upload MTemp1.wsp to destination site collection http://sbx08:3797/sites/MDestination solution gallery
• Create a sub site http://sbx08:3797/sites/MDestination/SubD2 using MTemp1.wsp template

   

• The interesting thing is the Shared Documents library created from the template has different behavior compare to the one migrated by powershell. The managed metadata columns on list are visible as shown in the screen shot!
  

It’s very clear at this point that export/import or backup/restore powershell commands did not handle managed metadata columns on Shared Documents library correctly! Of cause, after these managed metadata columns successfully migrated, we still need to relink back to the correct managed metadata term stores that will be published in future blogs.

Even more intresting is if we migrate sub site from DocAve and set append to the destination site, we got the same result as export/import. However, if we select to overwrite destination from DocAve, we got the managed metadata columns but NO data displayed for Shared Documents as the following screen shot. The managed metadata columns are migrated and linked correctly using DocAve to migrate sub sites with same MMS!

If you edit the items inside Shared Documents, you are able to see the columns that is better than export/import process! We are checking with DocAve to see whether we could migrate the managed metadata for Shared Documents library.

We have worked with Microsoft and it seems like this issue could not be reproduced from older versions such as June CU 2010 or SP1. But could be reproduced on Feb CU 2012 and June CU 2012.

Thanks for our SharePoint developer Anitha who is working closely with me to identify some issues and find ways to verify the results. She has provided a work around and we could share with you later.