Wednesday, May 18, 2011

401 UNAUTHORIZED error when integrate Perl from Window server to SharePoint 2010 site through SOAP web service

We have engineers using perl on Linux and Unix running testing scripts. They would like to send the testing result directly from perl to SharePoint. The perl to SharePoint integration is critical for our engineers on their daily jobs.

Since we need to pass authentication, we are trying to pass this on window server first before moving to Linux or Unix. We have identified two good blogs on this topics and tried the approach.  However, we are getting 401 UNAUTHORIZED error against both NTLM and Kerberos SharePoint 2010 sites. We believe the NTLM perl package may not configured correctly or Perl may not connect to SharePoint Kerberos sites. Since we are not perl or SharePoint integration expertise, we could like to get some help.


We have tried on 2008 64bit window VM server before moving to Linux or Unix. Here are the steps we tried.
  1. Install ActivePerl 5.12.3
  2. Install SOAP::Lite using cpan command cpan -i SOAP:Lite since it’s difficult using ppm
  3. Setup SharePoint site and list and grant site collection admin to the test user account
  4. Tried the script from The SharePoint / Perl Connection (Part 1)
  5. Error The exception is on line: $lists = $soap->GetListCollection();
          401 Unauthorized at E:\Share\ws\ws.pl line 27
          Client-Warning: Unsupported authentication scheme 'ntlm'.

    6. I also tried to add the following line:
          use Authen::NTLM; ntlmv2(1);
          But got error
          E:\Share\ws>ws1.pl
          Can't locate Authen/NTLM.pm in @INC (@INC contains: C:/Perl64/site/lib C:/Perl64
/lib .) at E:\Share\ws\ws1.pl line 3.

It seems like the NTLM.pm is inside C:\Perl64\lib\LWP\Authen.

    7. I also tried to install LWP::Authen::NTLM from CPAN http://search.cpan.org/~gaas/libwww-perl/lib/LWP/Authen/Ntlm.pm. I’m not familiar the perl make installation.


Here is the script from John Wefler and I modified with our server and site.
================================================
print "Start\n";

use LWP::UserAgent;
use LWP::Debug;
use SOAP::Lite on_action => sub { "$_[0]$_[1]"; };
import SOAP::Data 'name', 'value';
sub SOAP::Transport::HTTP::Client::get_basic_credentials { return ('user' => 'password') };
our $sp_endpoint = 'http://sbx01/sites/Harry/_vti_bin/lists.asmx';
our $sp_domain = 'sbx01.na.qulacomm.com:80';
our $sp_username = 'adminaccount';
our $sp_password = 'password';

$debug = 1;
if ($debug) {
    LWP::Debug::level('+');
    SOAP::Lite->import(+trace => 'all');
}
my @ua_args = (keep_alive => 1);
my @credentials = ($sp_domain, "", $sp_username, $sp_password);
my $schema_ua = LWP::UserAgent->new(@ua_args);
$schema_ua->credentials(@credentials);
$soap = SOAP::Lite->proxy($sp_endpoint, @ua_args, credentials => \@credentials);
$soap->schema->useragent($schema_ua);
$soap->uri("http://schemas.microsoft.com/sharepoint/soap/");

$lists = $soap->GetListCollection();
quit(1, $lists->faultstring()) if defined $lists->fault();

sub lists_getid {
    my $title = shift;
    my @result = $lists->dataof('//GetListCollectionResult/Lists/List');
    foreach my $data (@result) {
        my $attr = $data->attr;
        return $attr->{ID}
        if ($attr->{Title} eq $title);
    }
    return undef;
}
sub lists_getitems
{
    my $listid = shift;
    my $in_listName = name('listName' => $listid);
    my $in_viewName = name('viewName' => '');
    my $in_rowLimit = name('rowLimit' => 99999);
    my $call = $soap->GetListItems($in_listName, $in_viewName, $in_rowLimit);
    quit(1, $call->faultstring()) if defined $call->fault();
    return $call->dataof('//GetListItemsResult/listitems/data/row');
}
my $list_id = lists_getid('Shared Documents');
print "List ID is: $list_id\n";
my @items = lists_getitems($list_id);
foreach my $data (@items) {
    my $attr = $data->attr;
    print Dumper($attr);
}

print "END\n";

=============================================




Here is the error I got:

I also listed some reference I got from web for your reference.
Since the final goal is to use perl on Unix and Linux, we found LWP does not have NTLM implemented on Unix/Linux through CPAN. However, both wget and curl  ( command line web clients ) support NTLM, and curl supports Kerberos. So we tried to use perl with wget to communicate with SharePoint.

Here is an example command from RHEL 5 command line that gets results back from the Lists web service:

wget --post-file /tmp/listrequest --header 'Content-Type: application/soap+xml; charset="utf-8"' --http-user='NA\mchiles' --http-password="putyourpwhere"  http://sharepoint.qualcomm.com/_vti_bin/Lists.asmx

note that I put the file for the request in /tmp/listrequest.

If you save this xml body to this file,  it should work
##


<?xml version="1.0" encoding="UTF-8"?><soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope"><soap:Body><GetListCollection xmlns="http://schemas.microsoft.com/sharepoint/soap/"/></soap:Body></soap:Envelope>

##

We are still try to get more SharePoint functions done through perl and please let me know if we make some progress.



    5 comments:

    1. In case you are still on this problem there is a hack you can do to the NTLM and it works.

      http://shareperl.blogspot.com/2010/01/sharepoint-perl-connection.html

      ReplyDelete
    2. Hi Jason,

      Even i face with the same issue.Authorization failed(401).The hack which is provided in the link is for IIS 7 or 6. But i am having IIS 7.5 server similar to what Harry have.I couldnt find that line of code to be replaced in my whole hard disk indeed.

      Looking for some updates,

      Thanks
      Tony

      ReplyDelete
    3. I'm coming across the same difficulties: the 401 error. On the MOSS servers (both 2007 and 2010, IIS 7.5), I can see logon/logoff actions occurring, but need to dig a bit deeper with higher level of auditing.
      The test ntlm script does work.

      The thing is that it does not work from aix (6.1), linux (debian6, perl 5.10.1) and windows (x64, perl 5.12.4), but it does work from apple (lion). Same script, same version of main modules (SOAP::Lite, LWP::UserAgent). So I'm going to investigate a bit more.

      Cheers,
      Jeroen

      ReplyDelete
    4. Perl online training|Perl training|call us+919000444287 ...
      www.21cssindia.com/courses/perl-online-training-36.html
      Perl Online Training, Perl Scripting online training by real time Experts from Hyderabad, India. Call 9000444287 for online training demo. Online Perl training ...

      ReplyDelete
    5. Your blog has given me that thing which I never expect to get from all over the websites. Nice post guys!

      web developer

      ReplyDelete