Sunday, April 27, 2014

Uploading Documents and Setting Metadata Using SharePoint REST (One Version)

Technorati Tags: ,,

There are many examples of uploading documents using SharePoint 2013 REST/CSOM/JSOM and there are many issues. One issue is uploading documents into SharePoint Online with CSOM/JSOM. There is a 1.5mb limit. I work for a SharePoint ECM company and we have many customers that have documents much larger than 1.5mb. One way around this limitation is to use the SharePoint REST API which is limited  to 2gb. Just remember that REST requires you to post a byte array, and does not support reading from a stream. This can put a strain on memory resources.

Another issue is customers have versioning turned on and will complain that your solution creates two versions when uploading documents and setting metadata. This has always been a challenge when using SharePoint’s remote API. You can still use RPC which enables you to post the binary and the metadata in one call and only create one version. However, this can only be used from native apps and is limited to 50mb. You can upload and create only one version with CSOM/JSOM/REST by checking out the file before setting the metadata and then checking the file back in afterwards using the SPCheckinType.OverwriteCheckin. This works. However, if you try to check the file in and any field level validation fails, the check in fails. JavaScript code below.

function addFile() {   

getFileBuffer().done(function (result) {
upload(result.filename,result.content).done(function (data) {
var file = data.d;
checkOut(file.ServerRelativeUrl).done(function () {
updateMetadata(file.ServerRelativeUrl, null).done(function () {
checkIn(file.ServerRelativeUrl).done(function () { });
})
})

})
}).fail(function (err) {
var e = err;
});
}

function getFileBuffer() {
var file = $('#documentUpload')[0].files[0];
var fileName = file.name;
var dfd = $.Deferred();
var reader = new FileReader();

reader.onloadend = function (e) {
var result = { 'filename': fileName, 'content': e.target.result };
dfd.resolve(result);
}
reader.onerror = function (e) {
dfd.reject(e.target.error);
}

reader.readAsArrayBuffer(file);
return dfd;
}

function upload(filename, content) {
appweburl = decodeURIComponent(getQueryStringParameter('SPAppWebUrl'));
hostweburl = decodeURIComponent(getQueryStringParameter('SPHostUrl'));
var restSource = appweburl +
"/_api/SP.AppContextSite(@target)/web/lists/getbytitle('Documents')/rootfolder/files/add(url='" + filename + "',overwrite=true)?@target='" + hostweburl + "'";
var dfd = $.Deferred();

$.ajax(
{
'url': restSource,
'method': 'POST',
'data': content,
processData: false,
'headers': {
'accept': 'application/json;odata=verbose',
'X-RequestDigest': $('#__REQUESTDIGEST').val(),
"content-length": content.byteLength
},
'success': function (data) {
var d = data;
dfd.resolve(d);
},
'error': function (err) {
dfd.reject(err);
}
}
);

return dfd;
}
function checkOut(fileUrl) {
appweburl = decodeURIComponent(getQueryStringParameter('SPAppWebUrl'));
hostweburl = decodeURIComponent(getQueryStringParameter('SPHostUrl'));
var restSource = appweburl +
"/_api/SP.AppContextSite(@target)/web/lists/getbytitle('Documents')/rootfolder/files/getbyurl(url='" + fileUrl + "')/checkout?@target='" + hostweburl + "'";
var dfd = $.Deferred();
$.ajax(
{
'url': restSource,
'method': 'POST',
'headers': {
'accept': 'application/json;odata=verbose',
'content-type': 'application/json;odata=verbose',
'X-RequestDigest': $('#__REQUESTDIGEST').val()
},
'success': function (data) {
var d = data;
dfd.resolve(data.d);
},
'error': function (err) {
dfd.reject(err);
}
}
);

return dfd;

}
function updateMetadata(fileUrl) {

appweburl = decodeURIComponent(getQueryStringParameter('SPAppWebUrl'));
hostweburl = decodeURIComponent(getQueryStringParameter('SPHostUrl'));
var restSource = appweburl +
"/_api/SP.AppContextSite(@target)/web/lists/getbytitle('Documents')/rootfolder/files/getbyurl(url='" + fileUrl + "')/listitemallfields?@target='" + hostweburl + "'";
var dfd = $.Deferred();

$.ajax(
{
'url': restSource,
'method': 'POST',
'data': JSON.stringify({
'__metadata': {'type':'SP.ListItem'},
'Title': 'My Title 3'
}),
'headers': {
'accept': 'application/json;odata=verbose',
'content-type': 'application/json;odata=verbose',
'X-RequestDigest': $('#__REQUESTDIGEST').val(),
'X-Http-Method': 'PATCH',
"If-Match": "*"
},
'success': function (data) {
var d = data;
dfd.resolve();
},
'error': function (err) {
dfd.reject();
}
}
);

return dfd;

}
function checkIn(fileUrl) {
appweburl = decodeURIComponent(getQueryStringParameter('SPAppWebUrl'));
hostweburl = decodeURIComponent(getQueryStringParameter('SPHostUrl'));
var restSource = appweburl +
"/_api/SP.AppContextSite(@target)/web/lists/getbytitle('Documents')/rootfolder/files/getbyurl(url='" + fileUrl + "')/checkin?@target='" + hostweburl + "'";
var dfd = $.Deferred();

$.ajax(
{
'url': restSource,
'method': 'POST',
data: JSON.stringify({
'checkInType': 2,
'comment': 'whatever'
}),
'headers': {
'accept': 'application/json;odata=verbose',
'content-type': 'application/json;odata=verbose',
'X-RequestDigest': $('#__REQUESTDIGEST').val()
},
'success': function (data) {
var d = data;
dfd.resolve(data.d);
},
'error': function (err) {
dfd.reject(err);
}
}
);

return dfd;

}

Use the ValidateUpdateListItem Method


If your going to be setting or updating metadata using the SharePoint’s remote API, then I suggest you use the SP.ListItem’s new ValidateUpdateListItem method. This method is available only through the remote API and is new to SP2013. ValidateUpdateListItem  is very similar to the UpdateOverwriteVersion method available on the server API. ValidateUpdateListItem sets the metdata and if the bNewDocumentUpdate argument is set to true will call the UpdateOverwriteVersion method which will update without incrementing the version. This eliminates the need to make the extra calls to check out and check in the document. It also will check in the document if it is already checked out and use the checkInComment argument.  The method takes multiple SP.ListItemFormUpdateValue types as arguments. This type takes the internal name of the field along with a value.

function updateMetadataNoVersion(fileUrl) {
appweburl = decodeURIComponent(getQueryStringParameter('SPAppWebUrl'));
hostweburl = decodeURIComponent(getQueryStringParameter('SPHostUrl'));
var restSource = appweburl +
"/_api/SP.AppContextSite(@target)/web/lists/getbytitle('Documents')/rootfolder/files/getbyurl(url='" + fileUrl+ "')/listitemallfields/validateupdatelistitem?@target='" + hostweburl + "'";

var dfd = $.Deferred();

$.ajax(
{
'url': restSource,
'method': 'POST',
'data': JSON.stringify({
'formValues': [
{
'__metadata': { 'type': 'SP.ListItemFormUpdateValue' },
'FieldName': 'Title',
'FieldValue': 'My Title2'
},
{
'__metadata': { 'type': 'SP.ListItemFormUpdateValue' },
'FieldName': 'testautodate',
'FieldValue': 'asdfsdfsdf'
}
],
'bNewDocumentUpdate': true,
'checkInComment': ''
}),
'headers': {
'accept': 'application/json;odata=verbose',
'content-type': 'application/json;odata=verbose',
'X-RequestDigest': $('#__REQUESTDIGEST').val()
},
'success': function (data) {
var d = data;
dfd.resolve(d);
},
'error': function (err) {
dfd.reject(err);
}
}
);

return dfd;
}

The efficiency of this method is reflected in its return type which is a list of SP.ListItemFormUpdateValue . This type contains the validation response for each field being updated. You can use the ListItemFormUpdateValue.HasException property to check for errors then use the ErrorMessage property to log or inform the user.



I Prefer ValidateUpdateListItem


Below is the revised code of adding a file using the ValidateUpdateListItem method.

function addFile() {   

getFileBuffer().done(function (result) {
upload(result.filename,result.content).done(function (data) {
var file = data.d;
updateMetadataNoVersion(file.ServerRelativeUrl).done(function () {

})
})
}).fail(function (err) {
var e = err;
});
}

The ValidateUpdateListItem method eliminates extra remote calls and allows you to handle multiple validation errors. Using this method along with REST you can efficiently upload a document up to 2gb and create only one version. You can also use this method to update metadata without having to create new file. Just set the bNewDocumentUpdate argument to true and this will not increment the version.

9 comments:

Björn Stenberg said...

Hi Steve,

Thanks for your informative post on using ValidateUpdateListItem with REST. From what I can see it's the only one out there. Awesome!

I did find a typo though. The variable restSource contains:

var restSource = ... + "')/listitemallfields/validateupdatelistitem?@target='" + ...

I believe it should be:

var restSource = ... + "')/validateupdatelistitem?@target='" + ...

No listitemallfields and it works for me. Thanks!

/Björn Stenberg

Steve Curran said...

Thanks Bjorn. The listitemallfields represents the SPListItem object when working with a SPFile object. The validateupdatelistitem method is a method of the SPListItem object. If you are working with just list items and not a file then yes you do not want to use listitemallfields.

Anonymous said...

Hi Steve,
thank you very much for your post it was very helpful for me
my problem is with updating a lookup field single and multi
I have tried many things with no success could u help me in that

Thanks in advance
Suhad

Unknown said...

Hi Steve,
I want upload a file with metadata information like file uploaded location, customer info, category like (finance,SCM). Does it possible to achieve this in using File RestAPI or any other approaches?

Is there any search API's available to do an extensive search based on these customized metadata's to retrieve the file list from the server/cloud?

Anonymous said...

ValidateUpdateListItem method saved my day. Thank you!

Shwetank Sharma said...

Hi Steve ..
Superb post, I was looking for the same.

Thanks,
Shwetank

Muneeb said...

Hi Steve,
I am working on C# web application which is calling SharePoint via REST call using C# (no JavaScript). I am stuck in achieving two requirements:
1 ) I have a folder with meta data. I am able to update folder metadata but how I can update ALL document meta data inside that folder. they both sharing same Site Columns in folder and document content type.
2 ) How I can move the child folder from one parent folder to parent folder with all files within and with all properties/metadata. I have only one document library.

Please share the rest call if possible with all payloads.

Please advice.

Regards,
Syed Muneeb

Unknown said...

Hi Steve
Do you have experiences with Taxonomy fields and the ValidateUpdateListItem function?
I try to use it over csom and can’t get it to work. I don’t find a string representation for the term which is accepted by the ValidateUpdateListItem function. For example I tried this pattern "-1;#{term.Name}|{term.Id}".
Thanks in advance and kind regards
Lucas

Unknown said...

Hi Steve,

Thanks for your detailed post! I have tried to upload file and metadata using java, however metadata is not being uploaded when I am using 'cc1' as metadata and getting below error -
{"error":{"code":"-2147024809, System.ArgumentException","message":{"lang":"en-US","value":"Column 'cc1' does not exist. It may have been deleted by another user. /Shared Documents"}}}

The problem is only for those metadata whose encoded value is being returned. In my case, it is 'OData__x0063_c1'. Is there any way to upload a encoded metadata?

Post a Comment