Azure / Office 365 / SharePoint Development / Search

Configure a build and release pipeline for your SharePoint Framework solution deployments

May 15, 2017

In the previous article, I wrote how I achieved to setup a build and release pipeline to automate the publishing process of my SharePoint Framework app package and JS file to my environments.

Info: Use build and release pipelines in VSTS to automate your SharePoint Framework deployments.

In this article, I will focus on explaining the steps in detail how you can achieve it yourself.

Source control

It all starts with your source control system. For this I made use of a new Git version controlled project on Visual Studio Team Services. Once you have such a project, there is some configuration required in your project solution itself.

Gulp tasks configuration

The first step is to create the custom gulp tasks that will be used by the release definition on VSTS.

Update manifest task

The first task which I added to the gulpfile.js file is one to update the manifest JSON file. This file holds the CDN location to where you will upload your project files.

The task makes use of an argument cdnpath which you can pass as a command line argument. To call this gulp tasks, run: gulp update-manifest --cdnpath https://<your-cdn-location>.

Upload script files and solution package + deploy the latest solution package version

Next up is to create the two tasks for uploading the JS scripts files and solution package to SharePoint and another task for deploying the uploaded solution package to get it activated. For this, I made use of the approach I explained in the following blog posts: upload assetsupload app package and app package deployment.

One thing I changed to the gulp tasks is that it now also makes use of command line arguments which you can pass through by executing the gulp task. This way you can maintain the configuration of your environments within the build and release pipeline.

Here is are the steps to follow

  1. Install gulp-spsync-creds and node-sppkg-deploy as a developer dependency to your project: npm install gulp-spsync-creds node-sppkg-deploy --save-dev --save-exact
  2. Add the updated gulp upload tasks:

Info: whole content of my gulpfile.js. I also allowed you to specify these configuration parameters from within the gulpfile. So you can test things out publishing files to your development environment.

Info: these tasks are also available in the sp-dev-build-extensions repository: https://github.com/SharePoint/sp-dev-build-extensions.

Once these tasks are in place, the tasks can be executed with the following commands:

  • gulp upload-to-sharepoint --ship --username $(username) --password $(password) --tenant $(tenant) --cdnsite $(cdnsite) --cdnlib $(cdnlib)
  • gulp upload-app-pkg --ship --username $(username) --password $(password) --tenant $(tenant) --catalogsite $(catalogsite)
  • gulp deploy-sppkg --ship --username $(username) --password $(password) --tenant $(tenant) --catalogsite $(catalogsite)

Info: these commands already contain the variables which I am using in my release tasks. If you leave out these arguments, it will take the development configuration from within the file.

Build definition configuration

In your source control project on Visual Studio Team Services, go to the build & release section. There you will have the option to create a new build definition. When you create a new definition, you will be presented with a couple of templates. In the template option, you should have the NodeJS with Gulp template. This gives you a good start.

NodeJS with Gulp template

NodeJS with Gulp template

To the template, you should add two additional gulp tasks, which you can do by clicking on the add task link.

All build definition tasks

All build definition tasks

For the build definition, you should only have to configure the gulp tasks, all other tasks can be like they are configured by default.

Update manifest for the CDN - task configuration

The CDN update task needs to be configured as follows:

  • Gulp task: update-manifest
  • Arguments: --cdnpath your-cdn-location
Manifest task configuration

Manifest task configuration

Bundle the project - task configuration

The project bundling tasks needs to be configured as follows:

  • Gulp task: bundle
  • Arguments: --ship
Bundle task configuration

Bundle task configuration

Package solution - task configuration

The last task in your build definition is the package solution task. This one needs to be configured as follows:

  • Gulp task: package-solution
  • Arguments: --ship
Package solution task configuration

Package solution task configuration

Save this configuration, but it is not done yet.

Build definition trigger configuration

To automatically trigger the build process when you do a push to the master branch source control system, you should configure this under the triggers tab:

Automate continuous integration by enabling the trigger

Automate continuous integration by enabling the trigger

On the trigger page, just enable the continuous integration trigger. Once this is done, should do one more check.

Underneath the options tab, check that the default agent queue is set to hosted. If this is not the case, it might happen that the agent which is going to run the build process, does not support npm and gulp and eventually, the process will fail. When you selected the gulp template, this should already be configured correctly.

Hosted agent queue

Hosted agent queue

If not, change it and finally save these settings and your build definition should now be completed. If you want, you can test it out by pushing a new commit to your master branch. This should automatically trigger the build process to start up.

Release definition configuration

Once the build definition is set up, the last step is to configure your release definition.

Info: if you want, you can also configure everything from within the build definition itself. By splitting things in a build and release definition you have better control over your environment publishing process. Choice is up to you.

The first step is to go to the releases tab, which you can find under the Build & Release tab, and create a new release definition.

Create a release definition

Create a release definition

Best is to start with an empty template and click next. On the next page, you should choose the newly configured build definition as the source for to be used for the release definition. Also, check the continuous deployment option and click create.

Release definition configuration

Release definition configuration

Once the release definition is created, click on the environment menu and choose to configure variables.

Configure variables

Configure variables

By setting up variables for the release, it makes your commands easier as you can easily change them for all your environments. These are the variables you must create:

  • username: the username that will be used to upload the files to SharePoint;
  • password: the password for the account;
  • tenant: only the tenant name, will be used to concatenate your SharePoint URL: https://<tenant>.sharepoint.com
  • cdnlib: name of the CDN library on the CDN site;
  • catalogsite: relative path to your APP catalog site;
  • cdnsite: relative path to the site on which you CDN library is located. If it is the root site, you do not have to specify this.
Variables to configure

Variables to configure

Configure the extract files task

The first task you must add to the definition is the extract files task. Configure it as follows:

  • Archive file patterns: **/*.zip
  • Destination folder: $(System.DefaultWorkingDirectory)/build/release
Extract files task configuration

Extract files task configuration

This task will extract the contents of the generated build package to a release folder. This is also the location we have to link to in the gulp tasks.

Upload to SharePoint document library

Add a gulp task to your release definition and configure the task as follows:

  • Gulp file path: $(System.DefaultWorkingDirectory)/build/release/s/gulpfile.js
  • Gulp tasks(s): upload-to-sharepoint
  • Arguments: --ship --username $(username) --password $(password) --tenant $(tenant) --cdnsite $(cdnsite) --cdnlib $(cdnlib)
Upload to SharePoint document library task configuration

Upload to SharePoint document library task configuration

Upload app package to SharePoint

Add the task to upload the app package to the SharePoint app catalog site. For this you need to add another gulp task, and configure it as follows:

  • Gulp file path: $(System.DefaultWorkingDirectory)/build/release/s/gulpfile.js
  • Gulp tasks(s): upload-app-pkg
  • Arguments: --ship --username $(username) --password $(password) --tenant $(tenant) --catalogsite $(catalogsite)
Upload app package to SharePoint task configuration

Upload app package to SharePoint task configuration

Deploy the solution package

The last task to configure is the one to deploy the solution package in the App Catalog site. For this you need to add another gulp task, and configure it as follows:

  • Gulp file path: $(System.DefaultWorkingDirectory)/build/release/s/gulpfile.js
  • Gulp tasks(s): deploy-sppkg
  • Arguments: --ship --username $(username) --password $(password) --tenant $(tenant) --catalogsite $(catalogsite)

Other configuration steps

Right now, you did all the task configuration that is required for this release definition. To be able to run the release definition correctly, there are a couple of extra configuration steps.

Go to Artifacts and check the default created source and edit it and rename the source alias to build.

Set the source alias to build

Set the source alias to build

Go to the Triggers tab and set the branch in your continuous deployment trigger to your master branch (or any other if you are configuring it for other environments).

Set the trigger to the master branch

Set the trigger to the master branch

That was it, now your release definition is completed. After this step, it is time for you to test it out by pushing a new release to git and check that the build and release process get automatically triggered.

Updates

14/08/2017

Added the Gulp task to automate the deployment of the solution package.

Comments

  • Pingback: Office 365 – 2 wekelijks overzicht – deel 17 – SP&C NL()

  • Pingback: SharePoint Virtual Summit Takeaways; SharePoint Server 2016 and Beyond; Is Cloud Killing On-prem?()

  • Amal Abeygunawardana

    Thanks for the great article! How have you setup the Dev vs. Prod environments? Are they different tenants or different site collections in the same tenant? Because, SP Fx solutions are packages that get deployed to app catalog, can we have Dev Site Collection and Prod Site Collection in the same tenant?

    • Yannick Plenevaux

      Hello, unfortunately, I think this is not possible yet to have Dev, Test and Prod environments on the same tenant. However, according to what Vesa Juvonen states in the last PnP webcast https://www.youtube.com/watch?v=HHLhO7UVLik, Microsoft engineering is working on the capability to have a site collection scoped app catalog, I would think it will be a way to achieve it… I actually expect it too ;)

      • What Yannick is pointing out is indeed the case how it is at te moment. Although you could do development on the same test or production environment, because in development you are mostly going to run and debug in the workbench (online and offline). In that case you are not installing anything to the app catalog.

        But it is always best to have two environments so that you can safely do your development / testing. For this you can make use of the free one year developer tenant which you can request on: https://dev.office.com – on that tenant you can also activate the first release features. Which you probably deactivated on production.

        So if you have multiple environments and still want to automate, you can setup branches in source control. That way you can link the dev. branch to your dev. environment and prod. to the production environment

        • Yannick Plenevaux

          Totally agree about the workbench for DEV purposes. In my case, I am looking for something more such as a TEST/QA environment, for testing new features before going live. But well, it is true that tenant-scope PROD assets might be then accessed. Elio, if I get you, you recommend to split these env to different tenants ?
          My main problem here, is if I am right that we don’t have a separate package solution for each version, so we can’t even select if we want, for example, the TEST or PROD Webpart and that even in the hosted workbench when we want to test against real SharePoint data.
          I think, indeed, it will be the only valid solutions to have separate tenants.

          Well, I will check if I can re subscribe to a new dev tenant, my previous one expired and I used to work on our organisation one. (Way too many accounts between my customers, my personal and my organisation :D)

        • If you really want to do everything on one environment, you could. All you have to do is applying some changes to your package-solution.json before bundling and packaging.

          https://uploads.disquscdn.com/images/6ab7bd630e342dda04db7bd98533df2e0cd5a04753751bd567cec8fe67dcf5da.png

          So in the JSON file, update the name, id, zip package name. So what I did in the screenshot above is I added “test” to the name and zippedPackage, and changed the last number in the id from 8 to 9. This is also something which can be automated by a gulp task. After these changes run gulp bundle --ship and gulp package-solution --ship and upload the app package to your app catalog. You will see that you get two entries of your app package:

          https://uploads.disquscdn.com/images/bcd549c81b9e2b0836b6edf1ed0623f63325a58da1b1a95819d135a26d881868.png

          I would still recommend to use a test and prod. library for your assets, just to be sure you do not override anything from production.

        • Yannick Plenevaux

          Thks Elio, actually, it is exactly what I did and it works well but it is a manual and a bit rough trick, did not have the time yet to write a gulp task for it but why not…

          My concern is that I don’t want to go with a too deep work around with SPFx continously evolving. I would like to set up an automated build process like in your excellent POST (BTW, already live in my company for PROD only ;) ) . My automated process must be a solid, reliable and SPFx-upgrade-proof process, I am not sure I am not doing something different from the MS way of thinking and did not see much docs about it (actually, I think only yours ;) )

        • Marc

          In many cases clients do not have more than one Tenant so it is a challenge that has to be worked through.
          I have taken Elio’s fantastic approach and have created a Gulp Task to update the package-solution.json. You also need to create a gulp task that renames the WebPart ID in the WebPart’s manifest otherwise you will get an error when deploying.
          I have made a single Build Definition that creates three zip files (one for Dev, UAT and PROD) so that they can be used as a single artifact for Release to the different environments.
          This method works whether you are deploying to multiple Tenants or a single Tenant.

        • That is correct Marc, that is also the reason I created another gulp task in the past to allow environment configuration. I have shared it here: https://www.eliostruyf.com/gulp-task-to-change-environment-settings-in-sharepoint-framework-projects/

          The task allows you to create new environments and it will automatically update all the references. All the configuration per environment is stored in a separate configuration file.

          It has been updated this week to support the latest v1.2.0 release which has an updated configuration file structure.

  • Shawn F.

    Issue I’m seeing is with the release environments variables. They don’t seem to pull into the gulp task correctly. I can see in the out put the call is displaying the variable values, but when I out config.args[‘tenant’] I’m getting https://undefined.sharepoint.com

    • Something seems wrong if you get a SharePoint URL. Config.args[‘tenant’] should only return your Office 365 tenant name. Could it be that there is some misconfiguration in the arguments?

  • Hi Elio,

    I have successfully configured continuous deployment as per your guidance.

    But I am getting the below error when the build task is executed. I am very new to the continuous deployment in VSO and this may be a easy fix. Could you please advice?

    No agent could be found with the following capabilities: npm, node.js

  • I guess bundle –ship is not working
    Could you please explain about disabling warning in tsling file?

    • Under the “control options” of the gulp task, you can check the “continue on error” box.

      https://uploads.disquscdn.com/images/d7ed04883b2727091092abd8128c787e6b2a4b95a4a5166f2b905a62bf5ac7e6.png

      Of course if there is a real error in your code it will still go and package the solution, but that task will eventually fail. So, only configure it for the bundling.

      • Also when you do this, you will see the the status of the build job will change from succeeded to partially succeeded.

        https://uploads.disquscdn.com/images/41ad110bb651cce5aec60530039160d4e496e680d9fab9450fcbe8fc733ba720.png

        • Thank you Elio. It works.

        • Should I press the Deploy button by selecting the app in app catalog library for every code update?

        • Yes, but just found out a way to automate that part. Will create a new task for that.

        • Great, That is awesome Elio.

        • Just released the node-sppkg-deploy package. This can be used to automate the deployment. Would you be able to test it against your environment?

          https://www.npmjs.com/package/node-sppkg-deploy

          Install it with: npm install node-sppkg-deploy –save-exact

          At the bottom of the page you can find the code for the gulp task.

        • Sure, I am getting “build is not defined” error.

          Followed the below steps
          1) Ran npm install node-sppkg-deploy –save-exact
          2) Added sppkg-deploy in a js file
          3) added sppkg-deploy gulp task file referene in gulpfile.js

        • 1. Did you add it to the gulpfile.js file?
          2. Did you accidentally override the following line:

        • 1. Did you add it to the gulpfile.js file? – Yes – refer the code below
          2. Did you accidentally override the line const build = require(‘@microsoft/sp-build-web’); – No

          ‘use strict’;

          const gulp = require(‘gulp’);
          const build = require(‘@microsoft/sp-build-web’);

          require(‘./gulpfile-update-manifest’);
          require(‘./gulpfile-upload-app-package’);
          require(‘./gulpfile-upload-to-sharepoint’);
          require(‘./gulpfile-sppkg-deploy’);

          build.initialize(gulp);

        • Ok :-) if you have added it to a separate file, you will also have to add the build reference to the file:

        • That works :)

          gulp sppkg-deploy –username “userid” –password “password” –tenant “tenant” –catalogsite “sites/appcatalog”

          This command gives me an error

          ERROR: {“odata.error”:{“code”:”-2147024894, System.IO.FileNotFoundException”,”message”:{“lang”:”en-US”,”value”:”File Not Found.”}}}
          ERROR: The web ID and list ID could not be retrieved

        • My mistake :-), will do a quick update.

        • Sure, Elio

        • Update done, can you reinstall the package: npm install node-sppkg-deploy@latest –save-exact

        • It is working now. Thank you Elios.

          May we can create some reusable templates in VSTS to deploy in Office 365 CDN as well as other CDNs such as Azure, Akmai File shares. Thoughts?

        • Great, waiting on other feedback. Once I have that, I will announce it.

          When you want to use the CDN offering from Azure, you can make use of the deploy-azure-storage gulp task. So instead calling the upload-to-sharepoint task, you will be using that Azure storage one. Most probably it will also be available for other CDN offerings, but you will have to create the gulp task yourself.

        • Sure, Elio.

        • We may need to handle the first time app package upload.

        • Tested it on my site and you can automatically activate it via the node module. So normally no manual action is required. Are you experiencing something else?

        • Sorry my bad. There was a module installation issue. All looks good Elio.

  • Do we need to consider any rollback steps in this process for the worst case scenarios?

    • Every time you do a code change and generate a new bundle, the files will be unique. ATM they contain a unique key. These files will get referenced in the solution file. So if anything fails, nothing really harmful happened. From the moment the latest version of the solution package gets activated / deployed, it will use the references to the new files.