Cannot connect to O365 through remote powershell for global account w/ MFA enabled

Recently our company changed on-premises AD user’s UPN from userid@domain.company.com to userid@company.com. This will keep the UPN same as email so users could login to window server and O365 consistently using same login. We have MFA enabled for accounts and using ADFS for authentication. After the on-premises AD user’s UPN change, we have run the DirSync and verified users’ UPN synced to O365. However, I cannot login as the new UPN userid@company.com that is global administrator to O365 from remote powershell.

We have been working with Microsoft for several weeks and here are the findings and tips to debug such issue and the workaround.

1. The first thing to debug this issue is to confirm the user’s UPN value among on-premises AD, Azure AD, and O365. Here are powershell command to get UPNs. ALL the UPNs should be same.

# Get on-premises AD UPN

Get-ADUser -identity userid

# Dump all AD user information if needed

C:\Users\spsvc15>set l

LOCALAPPDATA=C:\Users\spsvc15\AppData\Local

LOGONSERVER=\\NADCSANxx

C:\Users\spsvc15>ldifde -f user.ldf -s nadcsan13 -d "CN=harryc,OU=Users,OU=San Diego,DC=na,DC=company,DC=com"

Connecting to "nadcsanxx"

Logging in as current user using SSPI

Exporting directory to file user.ldf

Searching for entries...

Writing out entries...

3 entries exported

The command has completed successfully

C:\Users\spsvc15>notepad user.ldf

# Get Azure AD UPN

Get-MsolUser -UserPrincipalName userid@company.com |fl

# Get SharePoint online user profile UPN

# Check details from the blog http://sharepointconnoisseur.blogspot.com/2015/02/different-ways-to-manage-o365-through.html

$userProperty = $UserProfileWS.GetUserPropertyByAccountName($login, 'SPS-UserPrincipalNam')

$currentUPN = $userProperty[0].Values[0].Value;

You could login to O365 and check active users with the userID and see user name also.

 

2. Second is to verify whether the account is MFA enabled.

Get-MsolUser -UserPrincipalName userid@company.com |fl 

Here are the example attributes that will indicate whether MFA is enabled for a user or not:

StrongAuthenticationRequirements       : {Microsoft.Online.Administration.StrongAuthenticationRequirement}

StrongAuthenticationUserDetails        :

StrongPasswordRequired                 : True

 

You could disable the MFA for specific user. Here are the details on how you can enable or disable MFA for a user in Office365:

http://blogs.office.com/2014/02/10/multi-factor-authentication-for-office-365/

https://msdn.microsoft.com/en-us/library/azure/dn383636.aspx

3. The third is to enable the remote powershell debug and verify the IDCRL traces. We would need to collect Powershell logs and IDCRL traces while reproducing the issue.  You could simulate the issue and collect the logs as mentioned below:

• PS Logs are by default always created and are available at: C:\Users\%username%\AppData\Local\Microsoft\Office365\Powershell 
• To enable and disable IDCRL logs please refer to the article below:  https://support.microsoft.com/en-us/kb/2433327

 

After we reviewed the IDCRL traces, we noticed the global account MFA enabled as in the below log.

---------------------------------

StrongAuthenticationMethod (2): <StrongAuthenticationMethod MethodType="5" Default="true" xmlns="http://schemas.microsoft.com/online/directoryservices/change/2008/11" />;

<StrongAuthenticationMethod MethodType="0" Default="false" xmlns="http://schemas.microsoft.com/online/directoryservices/change/2008/11" />

StrongAuthenticationProofupTime (1): 635578472822721952

StrongAuthenticationRequirement (1): <StrongAuthenticationRequirement RelyingParty="*" State="0" RememberDevicesNotIssuedBefore="0001-01-01T00:00:00Z" xmlns="http://schemas.microsoft.com/online/directoryservices/change/2008/11" />

StrongAuthenticationUserDetails (1): {<###ongAuthenticationUserDetails ###neNumber="+# ###5278057" ###ns="###p://###emas.###rosoft.###/###ine/###ectoryservices/###nge/###8/##" />}

---------------------------------

After we disable the MFA, we the error becomes different and it displayed below.

2015-05-16T07:04:33.4144824-07:00         Info        ConnectMsolService.MsolCmdletProcessRecord              f4ec3e9a-89e1-4f39-b6f2-44b0d9a32865   Getting Live Token for user with Credential = userid@company.com, FederationProviderId = microsoftonline.com, BecWebServiceLogonSiteName = ps.microsoftonline.com, Policy = MCMBI, BecWebServiceUri = https://provisioningapi.microsoftonline.com/provisioningwebservice.svc

2015-05-16T07:04:33.8357062-07:00         Exception            Connect-MsolService.Checking for retry               f4ec3e9a-89e1-4f39-b6f2-44b0d9a32865   Microsoft.Online.Administration.Automation.WindowsLiveException - An error occurred during authentication. Please retry your operation. If this problem persists, contact Technical Support..    at Microsoft.Online.Administration.Automation.LiveIdentityManager.GetAuthenticationStatus(String siteName)

   at Microsoft.Online.Administration.Automation.LiveIdentityManager.LogonPassport(String policy, String siteName)

   at Microsoft.Online.Administration.Automation.LiveIdentityManager.LogOnUser(String federationProviderId, String userName, String password, String siteName, String policy, String environment)

   at Microsoft.Online.Administration.Automation.ConnectMsolService.MsolCmdletProcessRecord()

2015-05-16T07:04:33.8981058-07:00         Info        MicrosoftOnlinePowershell                        f4ec3e9a-89e1-4f39-b6f2-44b0d9a32865   Connect-MsolService=12182ms

2015-05-16T07:04:33.8981058-07:00         Exception            MsolCmdlet.ProcessRecord                        f4ec3e9a-89e1-4f39-b6f2-44b0d9a32865         Microsoft.Online.Administration.Automation.MicrosoftOnlineException - Exception of type 'Microsoft.Online.Administration.Automation.MicrosoftOnlineException' was thrown..    at Microsoft.Online.Administration.Automation.ConnectMsolService.MsolCmdletProcessRecord()

   at Microsoft.Online.Administration.Automation.MsolCmdlet.ProcessRecord()

This seems to be a O365 bug and Microsoft finally confirmed with us that the root cause is O365 remote powershell does not MFA.

4. The forth is to verify whether you ahve proxy running between your Powershell server and the O365. You could use the foillowing command to verify.

C:\Users\spsvxx>netsh winhttp show proxy

Current WinHTTP proxy settings:

    Direct access (no proxy server).

The safer way to run remote Powershell is to use a local Azure global account with MFA disabled and run it on server with no proxy.