Why SharePoint 2010 search does not show some results?

SharePoint 2010 search is better than ever before. Enterprise search for SharePoint 2010 contains all the features and functionality of MOSS 2007 Search, like people search, but goes further with richer navigation, refinement and related search capabilities. After Microsoft acquired FAST 2 years ago and applied this to SharePoint, it now offers as a separate add-on to SharePoint for those willing to invest in high end enterprise search.

Since search is very comportment in SharePoint adoption, there are many cases users reported they could not find the expected items in the search result. I would like to share the tips we found since August 2010 so you could explain and resolve those "issues" quickly.

Before we dig into some specific search "issues", first you should take a look of the different SharePoint Versions Search Comparison before implement it in your company. Searches like People Search, Social, Taxonomy integration not included in Search Server 2010 Express. Second, verify search services have been setup and associated with webapp and crawling process has been completed without errors. Incremental  crawling process has also been scheduled. You could verify from central admin and refer some instructions. Now let's dig into some specific search "issues" users reported frequently.

1. Why documents or items did not displayed in my search result while is for some other people? This is very common questions people complain and most of the cases, it's the permission issue. Specific users may not have permission to read those documents and as a result, it will not be displayed in the search result.

2. Why documents on one site do not show in the search result but others shown?
Besides the permission checking, this is typical site search disabled setting issue. Go to "Site Actions"->"Site Settings"->Site Administration"->"Search and offline availability" to verify "Allow this site to appear in search results" is set to yes as shown in the screen shot. This is site setting and site owners could hide the items on the site intentionally not to display in the search result.

3. Why search not showing any results for anonymous users?
This is similar site search disabled setting issue described above and you could change site setting and set "Always index all Web Parts on this site" to true.

4. Why some of my documents did not show in the search result?
Yes, this happens to several users and this may be related to Draft documents that are in document library that requires approval. You could check whether your document library enabled the approval process by going to "Document Library Settings"->"Version Settings" and whether "Require content approval for submitted items?" is set to yes as in the screen shots.

There is another interesting setting on the Document Library setting "Draft Item Security" that will impacting your search result.

Here is the case if you have file 1 has version 1.0 that is in published status and version 1.1 checked-in but not approved.

If  "Draft Item Security"is set to "Only users who can edit items", file 1 will NOT in any search result as designed since the crawling process will only index the latest version. In this case, the latest version has not been published and as a result, this file will not be indexed and will not in search result.

However, people reported if "Draft Item Security" is set to "Any user who can read items", file 1 with version 1.0 is still in the index and will be displayed in the search result. However, my testing does NOT display this file in the search result that is consistent with the golden rule - only latest published version will be indexed and if latest version is in draft version, none of the versions will be indexed as designed.

4. Why PDF files not show in the search result?
This is easy answer. You need to install PDF iFilter to index PDF files. You could follow Microsoft instruction to install it.

5.  Why social tagging and discussion not show in the result?
You may check whether jobs to index the social tagging are running correctly. The job for this is scheduled hourly and you could change it from central admin as administrator. The job is named as "User Profile Service - Social Data Maintenance Job" that is to Aggregates social tags and ratings and cleans the social data change log.

6. Why people search not show any result?
Besides you need to have check whether you are NOT running Search Server 2010 Express, this may be the mysite search index issue. You should included the MySite in the index content as in the screen shot. You may refer the instruction listed in some bog and verify it.

7. Why no result show in the default simple search but show in advanced search?
If you can see your items in advanced search but not in default simple search, this might be the webapp zone setting bug described in MSDN discussion board we identified during testing. If you have multiple zones setup for the same webapp. Please mask sure the default zone is Intranet NOT other settings. This seems to be a bug you could not get search result if the default zone is NOT Intranet. We may need to submit this bug to Microsoft.

8. Why my external line of business data not show in the result even we have BCS setup already?
If you have setup BCS to bring external data to SharePoint through external content type and did not find those in search result, this might be the search setting issue. You could configure this in SharePoint Central Administration as described in previous blog.

9. Why documents or items did not displayed in my search result even I have the permission through SharePoint?
If you have implemeted some security policy like NextLabs and enabled Search Result Trimming, it will allows SharePoint to limit the display of search results to only those web parts or documents (i.e., list items) which the search user is authorized to view based on NextLabs entitlement policy. On the worlds, even users have been granted permission through SharePoint to have access those contents, they will not be able to see in search result if thay are restricted from NextLabs policy entitlement. This is one of the requirement from our security team to block groups to access the sesitive content. See my new blog for details.


There are some other search issues we may share later.

Risk to install Microsoft Office Web Apps for SharePoint 2010

Microsoft Office Web Apps is the online companion to Office Word, Excel, PowerPoint and OneNote applications that enables users to access documents from anywhere. Users can view, share, and work on documents with other users online across personal computers, mobile telephones, and the Web. Office Web Apps is available to users through Windows Live and to business customers with Microsoft Office 2010 volume licensing and document management solutions that are based on Microsoft SharePoint 2010 Products. You could find the installation with architecture design.

This feature is very helpful if you have SharePoint users accessing Microsoft office documents from UNIX and Linus systems without Microsoft Office client installed.
There is a big risk if you plan to install this to SharePoint 2010.  Our experience was that if an installation of Office Web Apps had to be uninstalled, the result would be that the server would be removed from the farm.  We experienced this with several environments in November 2010 and notified Microsoft.  Microsoft Product team confirmed on April 14, 2011 that the results we experienced are, by design, the correct behavior.  The explanation is that the Office Web Application service must be installed on all web servers and must stay in synch across those servers.  Therefore, to avoid a situation where the servers are not synched, the uninstall process removes the server from the farm.  

Microsoft mentioned this will not be resolved in the near term and may not in next SharePoint version (2015?). Microsoft offered a couple of work-arounds. Microsoft will send us a white-paper detailing those work-arounds. One of the work around is to disable Office Web Apps if installation failed. However,  the high risk is three is no clear procedure Office Web Apps could be upgraded to future version whence this issue been fixed.

The bottom line is that we consider the uninstall process and the work-arounds to be unacceptable risks to our environments and does not plan to use Office Web Apps in our SharePoint environments until an alternative low-risk uninstall process is provided by Microsoft.

SharePoint 2010 users missing pictures in SharePoint groups

We have found many SharePoint 2010 users do not have their pictures when they are added to local groups. Here is the common symptom with detailed description.

1.    Issue - User missing pictures in SharePoint
Picture  not  displayed when  user is added to a any SharePoint group (Members, Owners, Visitors etc. ) of a  SharePoint 2010 site.

2.    Description
When a user is added to any SharePoint group,  the picture does not show.
However,  Mysite has all 3 sizes of pictures (L,M,S) stored in  /User Photos/Profile Pictures/ . Attached are the organizational view of users. Each user has 3 sizes of pictures. My Organization also shows the correct pictures for those people.

The following screen shot shows properties of working picture where url is picked from mysite medium size picture(http://mysite/User%20Photos/Profile%20Pictures/na_harryc_MThumb.jpg).

The second screen shot shows properties of missing picture where url is pointing to person.gif picture(http://projects/_layouts/images/person.jpg). Please note projects is the webapp alias VIP where the site is located.

 
If you explicitly pasted the medium size mysite picture url of missing picture user in browser(http://mysite/User%20Photos/Profile%20Pictures/domain_userid_MThumb.jpg) , picture is displayed correctly. We could display users' organization with all pictures also. However, the SharePoint group seems not able to associate the users pictures with users.




3.    Procedure to Reproduce

• Open any SharePoint site in browser
• Navigate to site actions > site settings> people and groups
• Add new user to any SharePoint group
• Picture of a  user is not displayed

4.    Similar issues reported
You could find some people also reported this issue.

• http://social.msdn.microsoft.com/Forums/en-US/sharepoint2010general/thread/450a2c3e-9c82-4a85-839f-9f5d4834b59d
• http://social.technet.microsoft.com/Forums/en-US/sharepoint2010setup/thread/19c98bbb-eefa-4575-a6de-925f1f4fae37

We have some other reasons that could cause picture missing issues. Here is the partial list.

• Users do not have picture in AD
• Pictures not synced to AD
• Picture not slitted into three sizes.

Today (4/19/2011) I just found two jobs that might related to this issue. We may have to take a look of the job status.

•  User Profile to SharePoint Full Synchronization (Hourly) - Synchronizes user information from the user profile application to SharePoint users and synchronizes site memberships from SharePoint to the user profile application.

• User Profile to SharePoint Quick Synchronization (5 minutes) - Synchronizes user information from the user profile application to SharePoint users who were recently added to a site.

One update from Microsoft on May 6, 2011 is that this issue also reported by other customer and it may be fixed by Dec. CU. I may try to apply 2011 April CU and test it again.

Please let me know if you know how to debug or fix this issue.

SharePoint 2010 RSS Viewer Web Part Issues and Tips to display SharePoint list

After upgraded all SharePoint 2007 to 2010 in August 201, our users are excited to explore some new features and enhanced web parts. One of the enhanced web part listed in 2010 is RSS feed web part. It's quite simple when you display external public RSS feed. You just need to provide the URL to config the web part. However, there are several tricks you should be aware of before you could configure it if you need to feed SharePoint list into the web part. We are facing several issues and many people also reported some strange behaviors.

The first trick for RSS feed web part is you should change authentication provider settings from NTLM to Kerberos with AD SPN setup if you need to display any SharePoint list as RSS feed. You do NOT need to setup Kerberos if you just need to display external public RSS feed such as http://www.microsite.reuters.com/rss/businessNews. The way to setup is straight forward and have not changed since SharePoint 2007. You could check from central admin on each webapp as described here. I noticed if you do not setup AD SPN, the webpart is still not working. Here are the procedures to check whether Kerberosis in deed working for the webapp.

Check SPN setup using command setspn -l domain\installaccount. Here is the example output from the command. You should see three entries on the server and fully qualified name.

C:\>setspn -l na\installaccount
Registered ServicePrincipalNames for CN=,OU=CORP,OU=Service Accounts,DC=domian,DC=company,DC=com:
        http/servername
        http/servername.company.com
        http/servername.domian.company.com

In order to confirm the user is indeed login using Kerberos, you could open the Security logs and filter for “Event ID: 4624”. You will see Kerberos as Authentication Package. More details in the future if you are interested in this topic.

The second trick is you only need to setup Kerberos on the site that is using RSS feed webpart. The displayed site does not need to be Kerberos enabled. I have the screen shot below that has three RSS feed web parts. As long as the web part site is Kerberos enabled, it can display both Kerberos enabled or NTLM enabled site lists.

The third trick or issue is I'm not able to setup to auto refresh the web part with the latest feeds. It is still same as 2007 that it only refresh after IISRESET! I'm able to find the AJAX options and setup to enable Asynchronous Automatic Refresh and Update as in the below screen shot. However, the web part displayed error "An unexpected error occured while rendering the webpart.". There is even an typo in the error message!

As of today August 30, Microsoft has provided a workaround to refresh this webpart. Pleas refer to here for details on the workaround.

The fourth issue is I'm not able to display the feed across the farms. See previous third RSS feed web part. I'm getting error "The RSS webpart does not support authenticated feeds." that is same as you have not setup Kerberos.

There are some other issues you may encounter fro external RSS feeds with SSL and different sources. We have noticed people mentioned they could display feed across farm we will continue to debug.
 
Finally Microsoft confirmed this is not support to auto feed refresh to display SharePoint lists. However, we have identified a workaround that could refresh the webpart even it's not using Ajax. Please refer the workaround and let me know how it works.

If you have error "The RSS webpart does not support authenticated feeds." on SharePoint 2013 on-premises, you might need to sett the aspnet:AllowAnonymousImpersonation to false in web.config as Brandon mentioned. I would be cautious to change it before complete testing for other webparts especially Access apps, any BCS webparts, and custom webparts using security token.

If you try to display SharePoint online O365 RSS feed from on-premises even after you configure the trust, you will still get error like 'ProtocolError occurred trying to complete the request. The server returned a status code of : Forbidden and the status description is : "Forbidden"'. I will do more research how we can configure this.

How to enable SharePoint 2010 Web Analytics "Custom Report"

After we evaluated and put the design together to enable SharePoint 2010 web analytics reports in November 2010 as described in previous blog, we have seen many power users try to use the reports. However, we have identified several issues that is preventing end users to get meaningful reports.

The first mystery to us is SharePoint 2010 provides a "Custom Report" users could schedule to generate the customized reports based on users criteria.  However some site collection reports such as "Top Pages" is missing "Custom Report" button on the ribbon. We have identified this is only happening on some sites for some environments not all sites. You could refer the two pictures with and without "Custom Report" button. listed in previous bog.

While debugging this inconsistent issue across different environments and different sites, we have finally identified the issue. Here is the summary of root cause, procedure to verify, and command to to fix it.

1. Root cause: The root cause is there is hidden site feature named WACustomReports needs to be enabled in order for users to see "Custom Report" button. You could check this from 14 hive directory
C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\TEMPLATE\FEATURES\WACustomReports.

<Feature  Id="AF6D9AEC-7C38-4dda-997F-CC1DDBB87C92"
          Title="$Resources:wacore,WACustomReportsFeatureTitle"
          Description="$Resources:wacore,WACustomReportsFeatureDesc"
          Version="14.0.0.0"
          Hidden="TRUE"
          Scope="Site"
          SolutionId="7ED6CD55-B479-4EB7-A529-E99A24C10BD3"
          ReceiverAssembly="Microsoft.Office.Server.WebAnalytics.UI, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c"
          ReceiverClass="Microsoft.Office.Server.WebAnalytics.CustomReportsFeature"
          xmlns="http://schemas.microsoft.com/sharepoint/">
</Feature>

 2. Procedure to check. You could use the following powershell commend to verify whether you have this feature enabled on site collection.


Get-SPFeature -Site http://sitecollectionurl

If you have the feature enabled, you will see the following entry.
DisplayName                                               Id                                              Scope
-----------                                                   --                                               -----
WACustomReports                af6d9aec-7c38-4dda-997f-cc1ddbb87c92     Site


3. Commend to fix this if not already activated. Here is the commend to enable this feature so users could have the "Custom Report" button.


Enable-SPFEATURE WACustomReports –URL http://sitecollectionURL


Now you should have the "Custom Report" you could enjoy to use. We were working with Microsoft how we could enable this for all sites automatically. There is a thread on this already from MSDN.

I looked at issue again this week  April 26 2011, I realized that this feature is automatically enabled for sites created from OoB site definition. However, if you have your own site definition or template, you need to add this feature to the feature stapling to enable it after site created.

How to deploy saved site template in different SharePoint 2010 farm

As you know we could utilize the web template created by saving the site as template on SharePoint 2010. The use case listed here is that use created a site on their non-production SharePoint 2010 farm and saved as template. Use requested IT to help so other users could create site collections based on this template in production farm.

The way how to deploy this saved template for reuse is described before. The major issue left is site creation will fail if the features are not exactly same on the destination farm with template farm the template created from. There is a chance the non-production development SharePoint 2010 environment may have different features than production. When developers are testing this on their development box, they may not have all features developed to production either. How we could package the solution described in previous blog so it can be deployed and used successfully in the targeted SharePoint farm?

In SharePoint 2007, there is a tool STPInspector we could use to identify the mismatching features and repackage the stp file before the deployment. However, this tool no longer working against SharePoint 2010 wsp template. Someone will definite develop this tool shortly. At meantime, here are the way you could do to package the solution described in previous blog to a potable solution to avoid mismatching features.

• Create a site collection on destination SharePoint farm such as production environment using the same template such as “Team Site” user using to create the previous site.
• Save this template as template wsp file.
• Import this first wsp file from destination SharePoint farm into your visual studio  as described in previous blog.
• Open the ONet.xml file and find the site and web features from the <Configurations> tag.
• Import user wsp file from source SharePoint farm into your visual studio  as described in previous blog.
• Compare and replace the site and web features from the <Configurations> tag.
• Repackage the solution and change the scope as described in previous blog.
• You could deploy this to destination farm and create site collections based on this template without missing features error.

 Here is the partial of the ONet.xml for your reference.

    <Configurations>
        <Configuration ID="0" Name="Default" MasterUrl="_catalogs/masterpage/QualcommMasterElements/Qualcomm.master" ThemedCssFolderUrl="">
            <SiteFeatures>
                <!--DocumentRoutingResources Feature-->
                <Feature ID="{0c8a9a47-22a9-4798-82f1-00e62a96006e}" Name="FeatureDefinition/0c8a9a47-22a9-4798-82f1-00e62a96006e" SourceVersion="14.0.0.0" />
                <!--EnhancedTheming Feature-->
                <Feature ID="{068bc832-4951-11dc-8314-0800200c9a66}" Name="FeatureDefinition/068bc832-4951-11dc-8314-0800200c9a66" SourceVersion="14.0.0.0" />
                <!--Ratings Feature-->
                <Feature ID="{915c240e-a6cc-49b8-8b2c-0bff8b553ed3}" Name="FeatureDefinition/915c240e-a6cc-49b8-8b2c-0bff8b553ed3" SourceVersion="14.0.1.0" />
                <!--WebPartAdderGroups Feature-->
                <Feature ID="{2ed1c45e-a73b-4779-ae81-1524e4de467a}" Name="FeatureDefinition/2ed1c45e-a73b-4779-ae81-1524e4de467a" SourceVersion="14.0.0.0" />
                <!--Other Feature -->               
            </SiteFeatures>
            <WebFeatures>
                <!--WikiPageHomePage Feature-->
                <Feature ID="{00bfea71-d8fe-4fec-8dad-01c19a6e4053}" Name="FeatureDefinition/00bfea71-d8fe-4fec-8dad-01c19a6e4053" SourceVersion="1.0.0.0">
                    <Properties>
                        <Property Key="OldWelcomePage" Value="" />
                    </Properties>
                </Feature>               
                <!--WebPageLibrary Feature-->
                <Feature ID="{00bfea71-c796-4402-9f2f-0eb9a6e71b18}" Name="FeatureDefinition/00bfea71-c796-4402-9f2f-0eb9a6e71b18" SourceVersion="1.0.0.0" />
                <!—Other features-->
            </WebFeatures>
        </Configuration>
    </Configurations>


If you want to list the features on Farm, Site Collection and Site before you repackage the wsp file, you could use the powershell the discrepancy of the feature on different farms.

To see all features for a Site Collection:

• Get-SPFeature -Site http://sitecollection

To see all features for a Site:

• Get-SPFeature -Web http://siteurl

You could check features on farm and webapp using commands posted.

Now, you got the idea how to repackage the saved template so it can be deployed and used on different SharePoint farm.

There one issue on the sites created from such site template - the sites could NOT be backed up using Site Recycle Bin tool (Previous Site Deletion Capture) on SharePoint 2010 just discovered. I have described in the another blog http://sharepointconnoisseur.blogspot.com/2011/04/tips-and-tricks-for-site-recycle-bin.html.

How to save a site template and use it is the farm to create site collections and sub sites in 2010

In 2007 when you wanted to create a new Site Collection with a template saved from a site, the site template needed to be installed globally before it could be utilized. The command to  do this are:

       // Add template to the global gallery so you could use any where
       stsadm -o addtemplate -filename Regions.stp -title Regions

      // You could check the template availability
      stsadm -o enumtemplates

Of case there are several major issues associated such as feature compatibility across environment and workflow will not working if save template with content. Of cause we have identified workarounds already last year to resolve those issues. I’m not a fun of site template and prefer to use site definition sue to the better maintainability.

SharePoint 2010 is little different and there are many ways to implement reusable templates.  To be specific using site template inn 2010, you can choose to load a site template to create sub sites or site collection at creation time. Here are two major use cases.


1.    The most user cases are using saved template as sandbox solution gallery in the solution to create sub site. The default wsp file saved is web scope and sandbox solution anyway. You could refer this blog on the procedure. It’s quite simple and I will not explain this in the blog.

2.    The second use case is users wants this template available to create site collection from site collection. In additional this template should be available globally to be provisioned by auto-provisioning tool such as SharePoint Site Provisioning and Governance Assistant (SPGA) tool we are using to automate the site creation and approval process.  In order to make it available in central admin when creating the site, there are some options and I’m listing the best option from our implementation to our end users. Again, as I mentioned before, I’m still prefer to use site definition unless we have to use site template sine it save some time.

There are several options to do this and some of them does not make sense to me and I was totally lost at beginning. You should avoid those if you need to use site template.

The first way to install template seems like to use powershell command Install-SPWebTemplate. Here are the procedure and issues.

   Command: Install-SPWebTemplate -path "E:/Harry/Temp1.wsp" -name "Temp1" -description "TestWSP" -Confirm:$false

    Error is : Cannot find or parse web template file E:/Harry/Temp1.wsp
    Result: You will not get template installed and I’m not sure what the usage of this powershell usage and could not figure out

The second way is to deploy the solution using  powershell command Add-SPSolution. Here are the procedure and issues.
   Commands:    Add-SPSolution -LiteralPath " E:/Harry/Temp1.wsp"
                            Install-SPSolution Temp1.wsp –GACDeployment
  Issue is :     Cannot find it from central admin to use for site collection creation
  Workaround:   You have to do some configuration to use it. See the blog for details.


The third way and I believe the best way inspired by the response from Mirjam’s response is to repackage the wsp using Visual Studio and deploy as farm solution. Here is the way to do this and you have much better control and maintainability. Here are the steps we implemented it.

1. Create a new SharePoint 2010 project using Visual studio
2. Selecting “Import SharePoint Solution Package” template
3. Browse site template WSP file you want to use and select “Deploy this WSP as farm solution” – This is critical to make it available in the farm
4. Select items you need as site template existed on this WSP to be packaged and click ok
5. Change the web template feature scope from site to farm so that we can see this template on site collection creation page. Be aware of multiple features and you need to select the right one feature
6. Add DisplayCategory="MyTemplate" on web template elements.xml file to include this template on your MyTemplate tab
7. Package this solution and deploy the using Add-SPSolution
8. Now open the Site collection create page, we can able to see “Temp1” template on MyTemplate tab as shown in below screen shot.

At this time, you should be able to use this site template to create new site collections or sub sites. However, there is potential issue that may prevent sites to be created successful - feature comparability. Unless you have exact same features installed on the farm as the site you saved as site template, you have have error feature not installed on the farm error when creating sites. I'll address this issue in later blog.

There is another issue on the sites created from such site template - the sites could NOT be backed up using Site Recycle Bin tool (Previous Site Deletion Capture) on SharePoint 2010 just discovered. I have described in the another blog http://sharepointconnoisseur.blogspot.com/2011/04/tips-and-tricks-for-site-recycle-bin.html.