Unified SharePoint 2013 on-premises and SPO site provisioning with SharePoint Framework (SPFx), Azure Logic Apps, Queue, Azure Function, and Azure Automation using Hybrid Runbook Worker

In previous session we have discussed Office 365 site self-provisioning with SharePoint Framework (SPFx), Azure Logic Apps, Queue, and Azure Function. In our company we are utilizing the same architecture and framework to provisioning BOTH SharePoint online site and SharePoint on-premises site. The architecture will also unify the site provisioning for SharePoint on-premises 2013, 2016, or 2019. Here is the architecture and detailed design for the implementation.

The architecture is almost identical to we implemented previous for SharePoint online. The only difference is we have added a Azure Automation using Hybrid Runbook Worker that could run the SharePoint on-premises site creation. The previous Azure function will call the Azure Automation webhook. 

The key part here is to configure the Azure automation Runbook like PowerShell through Hybrid Runbook Worker. This way the PowerShell from Azure automation can provisioning the on-premises site through on-premises site creation method. Here is the architecture of the Azure automation Runbook Worker.

If you have set up Azure automation Runbook Worker, everything is almost same as we configured previous for SharePoint online site provisioning process. The overall architecture diagram is listed below.

1. User enters site provisioning information from SharePoint Framework (SPFx) form on SPO site.
2. The new list item created will trigger the Azure Logic app to send to approval.
3. If the request approved, Azure Logic App will send the request to Azure queue with request ID that is list item ID. We have separate the two queues and if the request of for on-premises site creation, the request will be send to on-premises queue.
4. Azure queue will trigger the Azure function.
5. Azure function will be trigger by new message from the queue.
6. Azure function will call the Azure Automation through webhook. Azure automation will use Runbook worker to provision the site collection using on-premises web services. Then perform post provisioning steps.
7. Azure Automation will update the SPO site request list with correct status.
8. Logic app will read the SPO site request list status
9. Logic app will send the emaul notification on the site creation.

Since this process is almost identical to SharePoint online site provisioning process except the step 6 & 7 Azure function will call Azure automation. You could refer previous blog for other steps.

The step 6 is Azure function will call the Azure Automation through webhook. You could set up a Azure automation Runbook Webhook as described procedure here. The Azure automation Runbook will look likes as below. Please remember to select the Hybrid Worker for "Run On". You need to record the webhook URL after created and you may not able to get it later!

The you could call this Azure automation just like the call below.

#Import dll from library

Import-Module "D:\home\site\wwwroot\qcsbxssspqueseprocessPS\Modules\SharePointPnPPowerShellOnline\3.0.1808.1\SharePointPnPPowerShellOnline.psd1" -Global;

#Get trigger content

$requestBody = Get-Content $triggerInput -Raw | ConvertFrom-Json

#$itemID = $requestBody.ItemId

$siteTitle = $requestBody.SiteTitle

$siteURL = $requestBody.SiteUrl

$output_SiteTitle = "SITETITLE: " + $siteTitle

Write-Output $output_SiteTitle

$output_SiteUrl = "SITEURL: " + $siteURL

Write-Output $output_SiteUrl

try

{

#region constructing Webhook call body

$webhookurl = 'https://s2events.azure-automation.net/webhooks?token=<webhookToken>'

$body = @{"SITETITLE" = $siteTitle; "SITEURL" = $siteURL}

$params = @{

ContentType = 'application/json'

Headers = @{'from' = 'Harry Chen'; 'Date' = "$(Get-Date)"}

Body = ($body | convertto-json)

Method = 'Post'

URI = $webhookurl

}

#Invoking call

Invoke-RestMethod @params -Verbose

Write-Output "Call Invoked - awaiting updating list item"

# Todo: Waite and query the SharePoint

#Connect-PnPOnline -url $requestListSiteUrl -Credentials $creds

#Set-PnPListItem -List $requestList -Identity $itemID -Values @{"Provisioning_x0020_Status" = "Provisioned";}

Write-Output "Item Updated"

#end region

}

catch [System.Exception]

{

$output = "Error Details: "  + $_.Exception.Message

Write-Output $output

}

The step 7 is to use web service to create on-premises site. We tried SP App model in on-premises with just CSOM mentioned by Vesa Juvonen. However, we are running into some issues. As a result, we are using the on-premises admin web service "_vti_adm/Admin.asmx?WSD" to create the on-premises site.
The Azure automation code looks likes as below.

......

[CmdletBinding()]

Param(

[object]$WebhookData,

[string]$siteTitle,

[string]$siteUrl)

if ($WebhookData)

{

Write-Output ("Starting runbook from webhook")

# Collect properties of WebhookData

$WebhookName = $WebHookData.WebhookName

$WebhookHeaders = $WebHookData.RequestHeader

$WebhookBody = $WebHookData.RequestBody

# Collect individual headers. Input converted from JSON.

$From = $WebhookHeaders.From

$InputBody = (ConvertFrom-Json -InputObject $WebhookBody)

Write-Verbose "WebhookBody: $InputBody"

$url = $InputBody.url

$fileName = $InputBody.fileName

Write-Output -InputObject ('Runbook started from webhook {0} by {1}.' -f $WebhookName, $From)

$siteTitle = $InputBody.siteTitle

$output = "Site Title is: " + $siteTitle

Write-Output $output

$siteUrl = $InputBody.siteUrl

$output = "Site Url is: " + $siteUrl

Write-Output $output

} else

{

Write-Output ("Starting runbook manually")

Write-Output ("Input Parameters following:")

$output = "Site Title is: " + $siteTitle

Write-Output $output

$output = "Site Url is: " + $siteUrl

Write-Output $output

}

try

{

$user = "owner"# Owner Login

$userName = "owner"# Owner Name

$pwd = 'password'

$adminSiteUrl = "http://SPadmin:port#/"

$securePwd = ConvertTo-SecureString $pwd -AsPlainText -Force

$cred = New-Object PSCredential($user, $securePwd)

$wsdlUrl = $adminSiteUrl + "/_vti_adm/Admin.asmx?WSDL"

$svc = New-WebServiceProxy -Uri $wsdlUrl -Credential $cred

$output = "Before Running Web Service - Site URL:" + $siteUrl + " Site Title: " + $siteTitle

write-output $output

$svc.CreateSite(

$siteUrl, # URL

$siteTitle, # Title

"", # Description

1033, # LCID

"STS#0", # WebTemplate

$user, # Owner Login

$userName, # Owner Name

"", # Owner Email

"", # PortalUrl

"") # PortalName

# Todo: Udpate the SPO list status w/ Completed status

# Post provisioning steps

}

catch

{

write-Output "`n[Main] Errors found:`n$_"

# Todo: Udpate the SPO list status w/ error status

}

You can see this is remote site provisioning process and the Azure Runbook worker can be any on-premises
server and does not need to be the SharePoint server. We have verified that the admin web service is supported
for SharePoint 2013, 2016, and 2019. So the code is same for all SharePoint versions.

At this point, we are using SharePoint farm account to provisioning the on-premises site. We are testing if
we could use another account to create site.

Now, we have combined both SharePoint online and SharePoint on-premises site creation into a same Azure
process!

Office 365 site self-provisioning with SharePoint Framework (SPFx), Azure Logic Apps, Queue, and Azure Function

About three years, when we first looked at the Office 365 site provisioning for the company, there were limited options. The best option at that time was using provider hosted SharePoint add-in architecture similar like posted here. Since then PnP provisioning package came out and has been gone through multiple iterations. With many Azure services available and integration with O365 now, there are more better options like the one published here. We have put together the Office 365 Modern Provisioning architecture and implemented using SharePoint Framework (SPFx), Azure Logic Apps, Queue, and Azure Function for the company. We will compare some alternative and options in the blog. We will cover SharePoint on-premises site self-provisioning with SharePoint Framework (SPFx), Azure Logic Apps, Queue, and Azure Function in another blog.

The Office 365 site self-provisioning architecture is illustrated in the below diagram. Here are the components.

  

• User enters site provisioning information from SharePoint Framework (SPFx) form on SPO site.
• The new list item created will trigger the Azure Logic app to send to approval.
• If the request approved, Azure Logic App will send the request to Azure queue with request ID that is list item ID.
• Azure queue will trigger the Azure function to provision the site collection and perform post provisioning steps.

Here is the details for each component and different alternative.

1. The first component is SharePoint online list that will store all the user site request. The reason we use this is to capture the site creation request for history and analysis. The list should have minimal the following fields. You could adjust fields based on your business requirement.

• SPSiteTitle
• SPSiteURL
• SPSiteType
• SPSiteDescriptions
• SPSiteOwners
• SPSiteMembers
• SPSiteVistors
• ApprovalStatus – Capture the approval status
• ProvisioningStatus – This will be used to indicate if site create successfully

2. The second component is SharePoint Framework (SPFx) form. You could use Azure PowerApps, blot, SharePoint add-in as alternatives. The reason we select SPFx form because we need extensive field validation. One example is when user enters the site URL, we need to verify if the URL is already in SharePoint or already in the request list. Our current SPFx form looks like the screen below.

3. The third component is the Azure Logic App. The Logic App will be triggered when new site request SharePoint online list item created. You could use Azure flow or WebJob as alternatives. The reason we select Azure Logic App because we have not open flow for users and we would like to control all components inside same place. In this case is inside Azure resource groups.

4. The forth component is the Azure storage queue. The queue has the site provisioning request information that will trigger the Azure function. The reason we choice the queue is to decouple the services and we will extend to support both  SharePoint online and SharePoint on-premises site provisioning. We decided to only include the "RequestId" in the queue to indicate the request list item ID. This will make the design flexible since any fields we need to add to the site creation will not impact the queue. The queue looks like this below. 

{

"RequestId": "86"

}

5. The firth component is the Azure function. We use PnP PowerShell as the implementation. You can use any language as the implementation but PnP PowerShell is much simpler. The only concern is PowerShell support is experimental at current V1.x. We are hoping future PowerShell V2.x will still be supported in Azure function. Here is the snippet of the Azure function code.

#Import dll from library

Import-Module "D:\home\site\wwwroot\yourmodulefodler\modules\SharePointPnPPowerShellOnline.psd1" -Global;

Import-Module "D:\home\site\wwwroot\yourmodulefodler\SharePoint Online Management Shell\Microsoft.Online.SharePoint.PowerShell" -Global;

#Get trigger content

$requestBody = Get-Content $triggerInput -Raw | ConvertFrom-Json

#Initial variables

$username = $env:sssp_username

$password = $env:sssp_password

$requestListSiteUrl = $env:sssp_requestsiteurl

$requestList = $env:sssp_requestlistname

$siteCollectionRootURL = $env:sssp_rooturl

$SiteDesign ="Topic"    #Right now hard coded to Topic only

$secpasswd = ConvertTo-SecureString $password -AsPlainText -Force

$creds = New-Object System.Management.Automation.PSCredential ($username, $secpasswd)

#Get request ID from queue

$requestID = $requestBody.RequestId

#Use request ID from the queue to retrive request information form the SPO site request list

    # Connect to request site and get request information based on request ID

    Connect-PnPOnline -url $requestListSiteUrl -Credentials $creds

    #Get request item

    $requestItem = Get-PnPListItem -List $requestList -Id $requestID

    $requestItem

    #Get all required site request information

    $requestSiteTitle =  $requestItem["SPSiteName"]

    $requestSiteUrl = $requestItem["SPSiteURL"]

    $requestDescription = $requestItem["SPSiteDescription"]

    $requestSiteOwners = $requestItem["SPSiteOwners"]

    $requestSiteMembers = $requestItem["SPSiteMembers"]

    $requestSiteVisitors = $requestItem["SPSiteVisitors"]

    $requestSiteType = $requestItem["SPSiteType"]

    #Create the site collection

    $siteCollectionURL = $siteCollectionRootURL + $requestSiteUrl

    $newSiteUrl = New-PnPSite -Type CommunicationSite -Title $requestSiteTitle -Url $siteCollectionURL -Description $requestDescription -SiteDesign $SiteDesign

    #Connect to the new created site collection

    Connect-PnPOnline -url $siteCollectionURL -Credentials $creds

    #Add additional site collection admin

    Add-PnPSiteCollectionAdmin -Owners additionalowner@mycompany.com

    foreach ($owner in $requestSiteOwners)

    {

        #Add-PnPUserToGroup -LoginName $ownerEmail -Identity $siteOwnersGroup

        if($owner.Email -ne $null -and $owner.Email -ne "")

        {

            Add-PnPUserToGroup -LoginName $owner.Email -Identity 3

        }

        else

        {

            Add-PnPUserToGroup -LoginName $owner.LookupValue -Identity 3

        }

    }

    # Other post provisioning steps …

    # Connect to request site and set ProvisioningStatus to "Provisioned"

There are few enhancements you could do. One example is  you can set up a app and use AppId and App Secret in the Powershell as below.

Connect-PnPOnline -Url <String> -AppId <String> -AppSecret <String>

The other two enhancements we are thinking are use site design and utilize PnP template. At this time, we are enhance the Office 365 site self-provisioning process to support both SharePoint online and SharePoint on-premises site creation process. We will share the architecture and implementation in the future.