You’ve heard of the term “DevOps.” You might even be a little annoyed with its ever-increasing appearance in conversations on the web.
Especially when the conversations are led by a “Ninja” or “Rock star.” But when broken into its three components--culture, process, and tooling—it becomes obvious that the processes of DevOps serve to benefit all IT and development organizations.
Let’s face it: The process of deploying new SharePoint applications is riddled with issues. It’s slow, and sometimes simply avoided for that reason alone. Thus end-users are missing out, and IT is being blamed for unsuccessful SharePoint.
I’d like to show you some ways to introduce DevOps-like processes for SharePoint application development without having to become a “Ninja” or a “Rock star.”
4 challenges with DevOps
There’s no doubt that organizations can benefit from faster delivery of higher quality SharePoint solutions. This is where the principles, and processes of DevOps can help. If you have been led to believe that DevOps is cookie cutter, and an all-or-nothing proposition, they led you wrong.
There’s nothing that prevents an organization from taking the process improvements of DevOps and implementing them in phases to help improve the speed and quality of the solutions your team provides the organization. Let’s look at the challenges of doing DevOps with SharePoint on-premises:
- The farm is the application. It’s hard to separate the solutions built for the farm from the farm itself. Therefore when you deploy applications, you’re reliant on existing instances of the farm. In order to live continuous integration and delivery, you really need to re-deploy the farm along with all the solutions for each test, and provide labs for each developer.
- Because of the above, SharePoint is not just a web application. It has a lot of moving parts. Even a small farm is a multi-tier application. But it also includes elements that are part of the entire organization such as Active Directory, and SQL Server. So when you look at the structure of the application, you’re considering several servers and integration points.
- Solutions on the farm often don’t belong to one team. However, each team has one or more solution that needs to be deployed into the broader farm. So how do various teams deploy their application on a shared farm in a DevOps kind of way? To add to this complexity, contractors are often used to do the development.
- SharePoint projects are set up in a waterfall manner. At least initially, projects are started by a need from a business user, a project is approved, a team assigned, and requirements gathered. This is a waterfall process. When implementation happens, you can quickly switch to an Agile development process or DevOps, but at least initially the planning stage is different. And the tendency is to keep it all waterfall.
4 ways DevOps can help
What we would all like to do is be able to have disparate solution development teams deploy their solution as often as they would like, to testing, staging, and production.
They might deploy to testing on a daily basis, staging near solution completion, and production, when everything is good. In most organizations, this might be every three to six months per solution.
If we accomplished the above, we could do the following:
- Break SharePoint features into smaller chunks, leverage new functionality, and develop features sooner.
- Test new functionality without fear of destroying the entire farm.
- Make our users happy with more features sooner, thus reducing SharePoint Rage.
- Waste less of IT’s time on provisioning for project teams and allow them more time for focusing on performance.
8 things you can do to make DevOps work
But how do you accomplish this? The secrets to bringing SharePoint solutions into the DevOps delivery pipeline can be summed up in these 8 tips:
Build containers for everything. If I were to emphasize anything, it would be the fact that you need to unshackle your SharePoint farms and move them to virtual machine (VM) containers, and vLANs or SDN technology. If you’re doing this already, great. But you might not realize the power of compartmentalizing the farm. Assuming you’re using a cloud with good storage capacity or have some on-premises orchestration tool, you can very easily give every developer his or her own instance of the farm to work on. Leverage great replication tools for SharePoint to make sure there is always a small farm version of your production farm; let’s call it the gold master. If you keep regular snapshots (nightly) of the gold master, then you can easily replicate it for every project. This will dramatically reduce issues from random environment variables. Keep your developers’ boxes small and clean. And decrease the completion time for solutions.
Break down project components into backlogs. Instead of looking at solution development as one large unit, have your teams break a project down into smaller components. Teams can then release components to staging on a more frequent basis. You can use some of the great backlog tracking tools to create this backlog, such as market leader Atlassian.
Get your users testing. Encourage users to give feedback on staging environments. Some organizations have been creative enough to build a reward system for business users to participate. Users in various departments become regular UAT testers. On a weekly or monthly basis, they provide these users with very basic test cases, and see how it goes.
Put code in a source depository. SharePoint code is still code, so put it in a source repository. No, GitHub is not intimately aware of WSPs and Windows PowerShell like it is with other languages. But the benefits of versioning and change control are well worth using a good source control system for all your solutions code.
Script your environments. This is a very hard habit to get into, but along with using containers, the platform-level implementation of your farm should be scripted out in PowerShell instead of manually configuring it each time. This way developers can use this script to get a base dev environment for their use. The key is to be proactive and lead with the script, not do it after the fact. If you already have a mature farm, there are some tools that can crawl your farm and get you started. Also know when to stop. If you go too deep, you will run into issues. Keep it to platform, or highest-level information architecture such as web applications. The fringe benefit of doing this scripting is it automatically becomes documentation!
Allow everyone to have an instance. If you do the above correctly, there’s no reason each developer can’t have an instance of a near-production development environment. The idea of throw-away environments is sometimes hard to get used to, but so powerful when you start doing it. By doing this, you allow the developers to test a solution against the entire farm, avoiding countless headaches. And you allow them to be opportunistic about what they have developed.
Use automation tools. The great testing and deployment tools are more complex to use with SharePoint, but not limited to bottom-up web applications. Leverage these tools for deployment and functional testing:
- QA Functional Testing: Functional testing tools such as the Selenium language are an important way to make sure that the experience of the users is as expected. Create baseline testing tools that are run every test iteration (during deployment to staging and production). The baseline scripts should be relatively static and have the primary function of regression testing. Specialized scripts should be created for each solution developed that is smaller but more specific to the new solution. The one gotcha here is that you need someone to write the scripts. In the case of SharePoint solutions, that task should probably belong to the operations team. There are a lot of great tools out there for front-end testing, and they all work with SharePoint.
- Deployment: Yes, you can. You can automate your deployment using things like Team Foundation Server (TFS). The key bullet one, containers. What you do for each deployment is to first copy an instance of the gold master snapshot we discussed above. After you do that, you copy to it your WSPs files. It helps to have a shared network drive for all deployment files, so that you can use a static reference in your PowerShell scripts; you can even store your scripts here. Run PowerShell to deploy and publish them. I know with certain farm-level feature development, this is an overly simplified example. But the point is, it’s possible. If you use IaaS or on-prem orchestration tools that have an API, you can automate this portion of copying from a snapshot as well.
Monitor and Share. Get all of that Unified Logging System (ULS), IIS stuff into a place where it can be easily analyzed. Today there’s no excuse for not monitoring all your logs. And if you do it correctly, you can build great dashboards to monitor the performance, and more importantly, the responsiveness and adoption (as this is one of the top SharePoint challenges) of each solution deployment. All of the great log analysis tools work here. The IOPS for ingesting ULS logs is high, but well worth it as this is a treasure trove of SharePoint specific monitoring data.
What About Office 365?
Office 365 shares all the same elements with on-premises SharePoint development but has many different details. We will have to reserve this for another post.
But in most cases, Office 365 development is easier except for when solutions are using farm-level services and features. In the case of Office 365, you’re dealing with more departmental-level solutions, and instead of IaaS, the approach is with PaaS.
DevOps Culture: What You Need to Know
There is one glaring element of DevOps that we haven’t addressed: the culture. You can’t make a big change in this area without making some waves with the people involved.
DevOps attempts to merge the IT operations with the development team. The result is either an operations team that belongs to IT with a responsibility to developers as users, or a DevOps team that usually belongs to development with power in IT.
In any case, the additional complexity in SharePoint projects occurs because development is done on a project basis and seldom by a dedicated team.
Therefore, a practical solution is to keep operations as part of IT, but morph it a bit. This puts the burden of the culture shift on IT operations, because they might not have been as involved in projects before.
IT as a Services Unit
One way to do this is to make this unit a services organization. Instead of mandating tools, the team can be opportunistic and create a library of technology it makes available to the development teams.
So when developers start a project, they can choose from the pre-configured library. For example, make available administration tools, workflow tools, web parts, IaaS public or on-premises.
If this team selects and vets these tools, they can unleash them to the various project teams.
One class of tools should be mandated, however. Those tools are the ones associated with the release process and monitoring. After all, the delivery pipeline has to be consistent because it’s a centralized, shared process. And the monitoring needs to be uniform so that everyone is on the same page when talking numbers.
Bottom line, the operations group doesn’t need to be rock stars, but they do need to be facilitators. It is also important they build in some transparency. For example, they can give access to the monitoring and provisioning tools.
If They Can Do It, You Can Do It
It’s not just a pipe dream. There are several ISVs and forward-thinking system integrators who are already implementing pieces of SharePoint solution continuous integration with their customers.
One of these is a little company by the name of Microsoft. It has completely transitioned in-house development, as well as implemented great tooling such as the custom build definitions for TFS, Front-End Testing tools in Visual Studio Online, and more robust API and remote development capabilities in SharePoint 2013.
Yes, I know DevOps can seem like a trendy thing for the cool kids, and often is contrary to the processes of traditional line-of-business applications. But the benefits it provides are good for everyone.
