Procedure to resolve the issue - cannot add SharePoint online list item with form does not exist error

We have found an issue that we could add new item to a SharePoint online list. The error is the form does not found. After debugging the issue, we identified that display item, edit item, and add item all have the same issue. Here is the details how to debug and fix the issue.

1. Verify three forms. If you understand how SharePoint works, you will know immediately that this should be the issue on SharePoint three form issue (NewForm.aspx, DidForm.aspx, and EditForm.aspx).  You can verify after open the site with SharePoint designer. 

2. Try to recreate the missing forms. The next step is to use SharePoint designer to add these three forms as described here. However, we got server error.

3. The next step is to migrate the same list from another place using Sharegate. The migration failed and has the following message.

"Unable to create web part 'Microsoft.SharePoint.WebPartPages.ListFormWebPart': A Web Part or Web Form Control on this Page cannot be displayed or imported. You don't have Add and Customize Pages permissions required to perform this action.. In Office 365, this is mainly caused by the farm setting "Custom Script" being deactivated."

Now we identified the root cause.

4. Enable the "Custom Script" on the site using the script described here. 

Connect-SPOService -Url https://mycompany-admin.sharepoint.com

Set-SPOSite -Identity https://mycompany.sharepoint.com -DenyAddAndCustomizePages 0 

5. The final step is either recreate the three forms from SharePoint designer or migrate content from another environment.

6. You might want to disable the "Custom Script" again suing Powershell.

Connect-SPOService -Url https://mycompany-admin.sharepoint.com

Set-SPOSite -Identity https://mycompany.sharepoint.com -DenyAddAndCustomizePages 1

We are not sure why the three forms lost in the first place but this blog will give you the procedure to fix the issue. 

Ultimate procedures to configure "Calendar E-mail Extension for Office 365" apps for SharePoint online

Calendar E-mail Extension for Office 365 sends invitations, handles responses and allows Exchange resources to be booked from a SharePoint calendar, making organizing your calendar as simple and efficient as possible. Here is the summary of the features.

• Send meeting requests from SharePoint calendars. Now it's possible to invite attendees directly from a SharePoint Online calendar.
• Book Exchange rooms and equipment. You can book rooms and equipment in Exchange directly from a SharePoint Online calendar.
• View status replies and attendance status. The SharePoint Online Calendar displays every attendee’s invite status, including their acceptance status, tentative acceptances or any new time suggested.
• Check availability of attendees and resources. You can check resources' and attendees' availability when scheduling an event directly from a SharePoint Online calendar.

One of the key feature we are looking for is to check the availability of the Exchange rooms and equipment. There are few tricks to make this work in SharePoint online. Here are the details to configured this correctly.

1. The first step is to add "Calendar E-mail Extension for Office 365" to SharePoint online site collection.

You could "Add an app" from the SharePoint online site collection. Here is the app you should add.

2. The second step is to enable the conflict checking scripting on the site collection. This has to be done before you configure the calendar! Otherwise you have to update each existing calendar and save it to pick up the conflict checking script! 

The two steps are:

a. Enable the custom scripts on the site. The SharePoint Online Management  Powershell script is like this below.

# Change this URL to match your admin site
Connect-SPOService -Url https://<yourdomain>-admin.sharepoint.com
# Enter username and password
# Change this URL to match your SharePoint site
Set-SPOSite -Identity https://<yourdomain>.sharepoint.com/sites/yoursite -DenyAddAndCustomizePages $false

b. Update Calendar E-mail Extension for Office 365 app permission. 

• Open the SharePoint online site where the add-in is installed and navigate to http://<SharePointWebsite>/_layouts/15/AppInv.aspx.
• Enter the following App Id (d4f43231-f66d-4f82-93b1-bd5a9d9945c1) and click on Lookup.
• In the Permission Request XML box, enter the following xml:

      <AppPermissionRequests AllowAppOnlyPolicy="true" >
        <AppPermissionRequest Scope="http://sharepoint/content/sitecollection/web"       

         Right="FullControl" />
      </AppPermissionRequests>

• Click on Trust it.
• Click on the Calendar E-Mail Extension for Office 365 add-in in the site contents page
• Click on Configure on each calendar list where the add-in is configured and save the settings again.

The details is listed in vendor's documentation here.

3. The third step is to add calendar and configure the resources.

Click Settings->Site contents->"Calendar E-mail Extension for Office 365", this will bring to configuration page like below.

After add a new calendar, click "Configuration" of the calendar. Enter the account or the shared mailbox account. The enter password to verify.

The you could configure the "Attendees" and "Resources". Please note you might need to add additional room and equipment mailbox for your application as in the below screenshot. Please select "Check Availability".

Then you would need to click save.

4. The forth step is to hide some fields users may not need to simplify the forms.

You could change the room to one of the following three options. This depends on your business requirement

• Drop-Down Menu
• Radio Buttons
• Checkboxes (allow multiple selection)

The procedure is to click "List Settings"->Click "Event (with email extension) Content Types->"Exchange Rooms and Equipment"-> Click "Edit column". Then select one of the three options.

If you need to hide some files from users, please refer to my previous blog for details.

5. The fifth step you might need to do is to disable users to save booking when there is a conflict.

The default behavior after you configure to check availability of the resource, when you try to save the booking, it will prompt the dialog box when there is a conflict.

You will noticed that there are two buttons "Don't save" and "Save Anyway" that do not make sense to allow users to save if there is conflict. 

Well, there is a way to remove these two buttons and here are the details. There are three places you need to make the change. The URLs are listed below for list named "PCP Booking 2".

• https://mycompany.sharepoint.com/sites/hc/Lists/PCP%20Booking%202/calendar.aspx
• https://mycompany..sharepoint.com/sites/hc/Lists/PCP%20Booking/NewForm.aspx
• https://mycompany..sharepoint.com/sites/hc/Lists/PCP%20Booking/EditForm.aspx
• https://mycompany..sharepoint.com/sites/hc/Lists/PCP%20Booking/DispForm.aspx

Go to "Settings" -> "Edit the page" and find the "SharePoint|sapiens Calendar E-Mail Extension Client Script" webpart->Click "EDIT SNIPPET".

Then append the following script to hide two buttons. Click "Insert" and save the page.

      <style type="text/css">

      input[value='Save Anyway'] { display: none; }

      input[value="Don't save"]  { display: none; }

      input[value="Don't save"] + input[value='Save'] { display: none; }

      </style>

5. The sixth optional step you might want to do prevent users to create past event.You could configure this in the lsit Validation setting.

Click the list settings->Validation settings. Enter the following formula to prevent past event to be created.

=[End Time]>Created

You could use another formula to prevent the past event to be modified.

=[End Time]>Now() 

5. The seventh optional step you might want to allow user to edit their events created by themselves.

You can go into the List Settings, Advanced Settings, and you should be able to set Read and Edit access to items that the user has created.

Now the  "Calendar E-mail Extension for Office 365" apps is ready on the SharePoint online site.

Easy way to hide SharePoint online event list some fields like “All Day Event” and “Recurrence”

When we create a new calendar that is an event list  on SharePoint online site, as default it will display fields like  "All Day Event" and "Recurrence" as in the below screenshot. In some cases, you may not want these fields and here are different ways to hide them.

1. The most common way is to use SharePoint designer to create new form and then remove the fields in the advanced design. This has been described here in details.

The major challenge is the designer is not reliable toll and you might constantly get the following error "could not save the list changes to the server" like discussed here. I'm always getting this error and could not resolve it quickly. This is the way I would NOT recommend you to try!

2. The second way is to use Powershell to set the fields like described here. The key Powershell command is like below.

    $field = $list.Fields["FieldName"]
    $field.ShowInNewForm = $false
    $field.Update()

However, I'm not able to find the equivalent PnP SharePoint online command. If you do, please let me know. This Powershell command seems only works for SharePoint on-premises.

3. The third way is to use the OoB UI with few tricks. Here is the details.

• Go to "List Settings" -> Advanced settings" -> Set “Allow management of content types?” to “Yes”.
• Click the content type used in the event list and click one field with link like “Category”. You will be able to make the field “Hidden”. 
• Two fields like “All Day Event” and “Recurrence” do not have direct link you could modify. You should copy the URL from previous "Category" field like this.

https://mycompany.sharepoint.com/sites/ittest/site1/lit/_layouts/15/ManageContentTypeField.aspx?ctype=0x01020028CF0843C06A41A39C183B3C72AF843700CA81E905638F3F458F75A6B452C36053&List=cb607f06-a7ac-465a-847e-5ab7f178ed16&Field=Category&Fid=%7B6df9bd52%2D550e%2D4a30%2Dbc31%2Da4366832a87d%7D

• You could now remove the "Fid" Parameter and replace "Field" Parameter value with either "fRecurrence" or "fAllDayEvent". 

https://mycompany.sharepoint.com/sites/ittest/site1/lit/_layouts/15/ManageContentTypeField.aspx?ctype=0x01020028CF0843C06A41A39C183B3C72AF843700CA81E905638F3F458F75A6B452C36053&List=cb607f06-a7ac-465a-847e-5ab7f178ed16&Field=fAllDayEvent

https://mycompany.sharepoint.com/sites/ittest/site1/lit/_layouts/15/ManageContentTypeField.aspx?ctype=0x01020028CF0843C06A41A39C183B3C72AF843700CA81E905638F3F458F75A6B452C36053&List=cb607f06-a7ac-465a-847e-5ab7f178ed16&Field=fRecurrence

Please note if you hide the “All Day Event”  on the content type as in method #3, the calendar cannot be displayed in the event webparts! The events will be empty.

4. The forth way is to use the javascript if this is allowed on the SPO site. Here is the details.

Add a content editor webpart on respective pages (NewForm.aspx/ DispForm.aspx/ EditForm.aspx) and use below jQuery code to hide SharePoint field. 

<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
<script>
$(document).ready(function(){
$('nobr:contains("Category")').closest('tr').hide(); //here we are hiding field named "Category"
$('nobr:contains("All Day Event")').closest('tr').hide(); //here we are hiding field named "All Day Event"
$('nobr:contains("Recurrence")').closest('tr').hide(); //here we are hiding field named "Recurrence"
});
</script>

This method will hide the fields and still allow the event webpart to display the calendar events. However, this is only applicable when script is allowed on the site. If not, you could enabled it.

Now you should be able to hide  “All Day Event” and “Recurrence” along with other fields you want to hide from end users.

There should be other ways like Rest/Graph API to hide, or may be SPFx customization you can try. 

Different ways to apply SharePoint field customizer extension to existing multiple columns

SharePoint Extensions can be deployed to SharePoint Online, and you can use modern JavaScript tools and libraries to build them. The Field Customizer Extension allows modifying the Views of the field in a list view. It can be used to override the field representation in the list. Field Customizer can be used with site columns or directly to the field inside a list. Microsoftarticle shows the detailed step to create the Field Customizer Extension but with limited instruction to apply it to existing list columns. Here are multiple ways to apply Field Customizer Extension to existing list columns.

The way to do this is to ClientSiteComponentId GUID attribute of Field Customizer for the existing field. There are few ways to implement this.

1. The first way is to use CSOM PowerShell to update ClientSiteComponentId attribute.

$siteURL = "https://mycompany.sharepoint.com/sites/Harry-PM-Testing"

$userId = "admin@mycompany.onmicrosoft.com"

$pwd = Read-Host -Prompt "Please enter your password" -AsSecureString

$creds = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($userId,$pwd)

$listTitle = "Order"

$existingColumnName = "SPFxExeColumn"

$ClientSideComponentId = "acd4d36c-a4ad-123c-a123-93862bd52123"

$ctx = New-Object Microsoft.SharePoint.Client.ClientContext($siteUrl)

$ctx.Credentials = $creds 

try{

    $lists = $ctx.web.Lists

    $list = $lists.GetByTitle($listTitle)

    $listItems = $list.GetItems([Microsoft.SharePoint.Client.CamlQuery]::CreateAllItemsQuery())

    $ctx.load($listItems)

    $ctx.load($list.Fields)

    $ctx.executeQuery()

    $field = $list.Fields.GetByInternalNameOrTitle($existingColumnName)

    $guid = [GUID]$ClientSideComponentId

    $field.ClientSideComponentId=$guid

    $field.update()

    $ctx.Load($field)

    $ctx.executeQuery()

}

catch{

    write-host "$($_.Exception.Message)" - foregroundcolore red

}   

2. The second is to use REST API like mentioned here.

3. The third easiest way is to use the PnP PowerShell like below.

$siteURL = "https://ionisdevops.sharepoint.com/sites/Harry-PM-Testing"

$listTitle = "Order"

$existingColumnName = "SPFxExeColumn"

$ClientSideComponentId = "acd4d36c-a4ad-449c-a312-93862bd52376"

  

Connect-PnPOnline -Url $siteurl  

$guid = [GUID]$ClientSideComponentId

Set-PnPField -List $listTitle -Identity $existingColumnName -Values @{ClientSideComponentId=$guid} -UpdateExistingLists

You can apply the same Field Customizer to as many existing fields as you like as long as the field has all the required schema as the the original list. The fields do not need to be the same type like "Number" since it will not matter.

If you need to apply other SPFx extension like custom action to whole tenant, please refer to other blog.

Approaches to enable modern site experience at the root of a SharePoint tenant

After Microsoft release modern site experience, there is a request for existing customers to enable such modern experience at the root of a SharePoint tenant. There are two different ways at this time.  You might decide based on your urgency and requirement.

The first way is traditional way as described in here. The approach is to delete the root site and recreate it.

The second way is to use Microsoft PowerShell Enable-CommSite to convert to modern site. However, as of today the command is still not available globally.

If  enabling modern site experience at the root of a SharePoint tenant is not urgent, I would recommend you might wait for the PowerShell to be release.

Procedure to add Full-width column SPFx webpart without fat banner, any horizontal margin or padding, or users' comments

Modern SharePoint pages support layouts that allow users to organize the information they present on their pages. When you add a custom webpart, the default behavior is like below screenshot.

There are some cases that users would like to have a pages modern SharePoint pages with custom SPFx webpart without fat banner, without any horizontal margin or padding, or users' comments as highlighted from the screenshot. This has been reflected in user voice. Here are the detailed steps to implement.

1. The first step is to remove the fat banner on the page like communication modern page. There are at least two different ways to implement this. The difficult way is to develop and deploy a custom SPFx webpart to hide it as described in Mikael Svenson’s blog.  Another easy way is to copy the home.aspx page and rename to your page. Then this page will NOT have the fat banner as described in Rajesh Sitaraman’s blog.

2. The second step is to implement the SPFx as Full-width webpart. This can be done quickly by adding the below configuration code to the YourWebPart.manifest.jason file\ as described in Microsoft’s blog.  

"supportsFullBleed": true,

Then you need to add this webpart as Full-width column webpart. You might need to go to the “Site Pages” library and then click “Site Page” on the ribbon in order to get the Full-width column webpart option for the page. Sometime you would not see Full-width column webpart option if you click “App Page”.

3. The third step is to hide the user Comments. This can be done quickly by following the instruction provided by Greg. 

The updated page with the same webpart now displays much clean as in the below screenshot.

4. The forth thing is to remove the "Feedback" button. This is similar to we described before to use SPFx webpart to hide it as described in Harjit’s blog. 

5. The final one is to remove the "Like" "Save for later" at the bottom of the page. We may also use SPFx with css to hide this as in previous session.

I've not implement step 4 or 5 yest but the page already look much clean. Hope this would help.

How Microsoft completely failed to protect SharePoint Online files with Office 365 labels and DLP

Microsoft provided a feature that users could apply Office 365 labels and data loss prevention (DLP) policies for SharePoint Online team sites with various levels of information protection. This looks exciting to design and deploy Office 365 labels and DLP policies for baseline, sensitive, and highly confidential SharePoint Online team sites highlighted in this Microsoft article.

One of the use case is to use SharePoint policy to label one document library as high sensitive and do not allow to share with external users. External users are no allowed to request access. After you configure the policy, the document in the library will have a small icon to indicate the policy. See the screenshot. This looks good so far.

However, after few days configuration, debugging, and discussion with Microsoft, this DLP feature seems to be funny and stupid that will fail completely! Here are few issues and I’ll provide the real life example to explain the details.

The first issue is after the label and policy applied, it may take one day to apply to SharePoint site and seven days to synced to the item. When you have policy enabled on the document library, any document uploaded will NOT have the policy until policy applied up to one day delay. In previous screenshot, you will see two documents do not have the policy applied at that time!

This is like a law officer like policeman add a "Stop" sign to the road. However, it will not take effect until a day later. Every new car arrived to this street will not see the sign until one day later. This is to every new car!!!

The second issue is user can edit the document properties and remove the label. See screenshot below.

This is like a law officer like policeman add a "Stop" sign to the road. However, driver can remove the stop sign! Every driver can remove the stop sign!!!

The third issue is user can edit the document properties and change the label to a different one event it is not assigned to this library. See screenshot above there are two labels.

This is like a law officer like policeman add a "Stop" sign to the road. However, driver can change it to a different sign like "Slow" sign! Every driver can change the sign to ANY other sign available!!!

Now you will understand why SharePoint Online files with Office 365 labels and DLP protection feature is so funny and stupid! I've raise the user voice in Microsoft user voice site.

How to fix SharePoint 2013 workflow missing issue on SharePoint online site?

If users could not find the out of box SharePoint 2013 workflow on SharePoint online site, here are two different ways to verify and fix the issue.

The first way is to click "Setting"->"Workflow Settings"->"Work Flows"->"Workflow Health" and then activate "The workflow service store feature" as described here.

If you would to fix on multiple site collections, here are the PowerShell script you can run against the site.

1. Install SharePoint online PnP PowerShell package.

Install-Module SharePointPnPPowerShellOnline

2. Check the WorkflowServiceStore site collection feature

Connect-PnPOnline -Url <Site URL>
Get-PnPFeature -Identity 2c63df2b-ceab-42c6-aeff-b3968162d4b1

3. Enable the WorkflowServiceStore site collection feature

Enable-PnPFeature -Identity 2c63df2b-ceab-42c6-aeff-b3968162d4b1 -Force

That's it.

Should we use Power or PnP PowerShell to manage the SharePoint hub site and association?

With SharePoint hub sit become available and more popular these days, there are some requests to create hub sites and associate some sites to the hub site. It would be idea to use PowerShell to automate the process. There are two PowerShell implementation, the regular SharePoint online PowerShell and PnP PowerShell. The difference of these two PowerShell has been described on Microsoft community.

However, when I try to use them and compare to the latest SharePoint tenant admin UI, both of them have limitation that would cause issues. I would like to explain the limitation and how to create hub sites and associate the site to hub.

The new SharePoint online admin has added a new feature that you could register the hub site through UI. It will allow you to enter the "Display Name" like "My Hub". The display name will be displayed in the UI and will be used in hub site association.

However, if you use PowerShell, there is no parameter we can pass the the command for "Display Name" . As a result, the hub site registered but it will not be easy to find from the SharePoint online admin center.

Another issue is the hub site created from PowerShell will take sometime like few minutes so the Add-SPOHubSiteAssociation command will not fail but will not associate the site to hub!

After working with Microsoft, there is a second PowerShell to set the hub site title. Here is the command.

Set-SPOHubSite -Identity <Site URL> -Title "My hub site"

At this time I'm thinking he best approach to manage the hub sites and association is like the procedure below.

1. Create hub site from using PowerShell
2. Set hub site title using using PowerShell

2. Associate the site with hub site from either SPO PowerShell or PnP PowerShell

3. Report the hub site and association from PowerShell.

Hope this would help.

PowerShell to create SharePoint online managed metadata terms from xml file

Our company started to SharePoint online governance plan and design the information architecture. One topic is to provide a global term for tagging and categorization. There are few good article to use PowerShell or C# to create terms from input file, however none of them can satisfy the requirement to create terms from Powershell with multiple levels of terms, and multiple termSet, and mutiple term groups.

I've modified the script we used previously for SharePoint on-premises and here is the code you could use for SPO.

#################################################################################
# Tod: Please update the following paremeters:
# $user        SharePoint admin user account like "harryc@mycompany.onmicrosoft.com"
# $siteUrl     SahrePoint onloine admin site url like "https://mycompany-admin.sharepoint.com/"
# Please install the SharePoint client components SDK - http://www.microsoft.com/en-us/download/details.aspx?id=35585 prior to running this script.

# Input Arguments
param([string]$userName,[string]$siteUrl)

[System.Net.ServicePointManager]::SecurityProtocol =[System.Net.SecurityProtocolType]::Tls12
Set-ExecutionPolicy -Scope Process -Confirm:$False -ExecutionPolicy Bypass -Force

#Add references to SharePoint client assemblies and authenticate to Office 365 site - required for CSOM
Add-Type -Path "../Assembly\Microsoft.SharePoint.Client.dll"
Add-Type -Path "../Assembly\Microsoft.SharePoint.Client.Runtime.dll"
Add-Type -Path "../Assembly\Microsoft.SharePoint.Client.Taxonomy.dll"

#Specify tenant admin and URL
#$user = "harryc@mycompany.onmicrosoft.com"
#$siteUrl = "https://mycompany-admin.sharepoint.com/"
$user = $userName
$siteUrl = $siteUrl
$password = Read-Host -Prompt "Please enter your password" -AsSecureString

# Import term xml file
[xml]$importxml = Get-Content "..\Config\InputTerms.xml" -Encoding UTF8
#$Import.InnerXml
if ($importxml -eq $null) {
  write-host "Term creation file content is empty!"
  return

}

try{
        #Bind to MMS
        $ctx = New-Object Microsoft.SharePoint.Client.ClientContext($siteUrl)
        $creds = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($user,$password)
        $ctx.Credentials = $creds
        $mms = [Microsoft.SharePoint.Client.Taxonomy.TaxonomySession]::GetTaxonomySession($ctx)
        $ctx.Load($mms)
        $ctx.ExecuteQuery()

        #Retrieve Term Stores
        $termStores = $MMS.TermStores
        $ctx.Load($termStores)
        $ctx.ExecuteQuery()

        #Bind to Term Store
        $termStore = $termStores[0]
        $ctx.Load($termStore)
        $ctx.ExecuteQuery()


        Foreach ($groupName in $importxml.Group.Name)
        {
            #Create Groups
            $group = $termStore.CreateGroup($groupName,[System.Guid]::NewGuid().toString())
            $ctx.Load($group)
            $ctx.ExecuteQuery()
            Write-Host "Create group - " $groupName;

            Foreach ($termSet in $importxml.Group.TermSet){
                #Create Term Sets
                $newTermSet = $group.CreateTermSet($termSet.Name,[System.Guid]::NewGuid().toString(),1033)
                $ctx.Load($newTermSet)
                $ctx.ExecuteQuery()
                Write-Host "Create termSet - " $termSet.Name;

                Foreach ($term in $termSet.Term){
                    #Create Terms
                    $newTerm = $newTermSet.CreateTerm($term.Name,1033,[System.Guid]::NewGuid().toString())
                    $ctx.Load($newTerm)
                    $ctx.ExecuteQuery()
                    Write-Host "Create term - " $term.Name;
                    if($term.Term -ne $null){
                        CreateChildTerms $ctx $newTerm $term
                    }
                }
          }
        }
}
catch [System.Exception]
{ 
    Write-Host "Exception:"  + $_.Exception.Message -foregroundcolor black -backgroundcolor yellow  
}


#############################################################
# Function to create sub-terms recursively
# $ctx        ClientContext
# $rootTerm   Parent term to create sub-term 
# $term       xml sub-term need to be created
#############################################################
function CreateChildTerms([Microsoft.SharePoint.Client.ClientContext]$ctx, [Microsoft.SharePoint.Client.Taxonomy.Term]$rootTerm, [System.Xml.XmlNode]$term)
{

       foreach ($childnode in $term.Term)
       {
             $addedTerm = $rootTerm.CreateTerm($childnode.Name, 1033, [System.Guid]::NewGuid().toString());
             $ctx.Load($addedTerm);
             $ctx.ExecuteQuery();
             Write-Host "Create child term - " $childnode.Name;

             # If sub-term need to be created, create recursively
             if($childnode.Term -ne $null){
                CreateChildTerms $ctx $addedTerm $childnode
             }
       }

}

There are many things we might need to development around terms like we did for SharePoint on-premises. Here is the list of the scripts in order to provide disaster recovery capability for terms.

1. Script to export the tenant term groups, termSets, and terms along with object GUID in xml file
2. Script to import the tenant term groups, termSets, and terms along with object GUID in xml file

We will keep you posted the progress on these scripts.