Update Management Processes via ManageSuite Services
Table of Contents
- Microsoft Update Overview
- Windows 10 Feature Updates
- Non-OS (Third Party) Updates
Microsoft Update Overview
Suite3’s ManageSuite Automated Update Delivery service is conducted via the coordination of three different processes: the Inventory Process, the Approval Process, and the Delivery Process. The following is a brief summary of the actions taken during each process to efficiently and effectively deliver regular updates for supported Microsoft operating systems and applications.
Patch Inventory Process
The Windows Update Agent in each managed server and workstation is direct by Suite3’s ManageSuite agent to retrieve from Microsoft’s update servers a list of needed updates for that system. After a list of available updates is collected by the agent, they are then gathered, logged, and sorted into approval categories and policies via our ManageSuite platform to await deployment instruction assigned during the Approval Process.
Microsoft generally releases new security-related patches on the Second Tuesday of every month, commonly referred to as “Patch Tuesday”. As part of a recent change in behavior, updates for Microsoft Office and other non-OS, non-server products are now released throughout the month. All updates are carefully reviewed by our staff for known issues and are usually approved for distribution. In addition to Microsoft’s own documentation, we review the National Vulnerability Database (http://nvd.nist.gov) and CERT’s resources at Carnegie Mellon (http://www.cert.org ) regularly for alerts about new threats and their impact.
Based on Microsoft’s recent history of patches categorized as “Security Updates” causing various functionality problems with customer resources, we have adopted the policy of waiting seven full days before re-researching and approving this category of patches for distribution. Security Updates introduced outside Patch Tuesday may also be subject to this seven day delay depending on the severity of the vulnerability.
OS updates classified as “Definition Updates” for anti-spam and/or Windows Defender products get automatically approved and distributed on a rolling basis due to the very low risk and impact of these updates.
We strongly recommend manual installation of all Cumulative Updates or Update Rollups for on-premise Microsoft Exchange servers due to the size and complexity of their installation, as well as the need for functionality verification after the update. We do not deliver these updates automatically due to the risk of unwanted service interruption.
An update of any category that has been determined to cause significant functionality issues either through testing or third party reporting may be delayed temporarily, if it is expected that a fix is forthcoming, or permanently if its potential functionality issues are determined to outweigh its benefits. Depending on the issues discovered, patches may also be withheld from certain operating systems or hardware platforms, but not others. Any updates that are temporarily delayed are reevaluated at least once every 90 days to determine if functionality issues have been solved. All updates that are permanently denied are nevertheless periodically reevaluated in case fixes to said patches were developed but not advertised. Please note that this is a very rare occurrence; of the thousands of patches that our staff has reviewed over the years, we have only marked a small handful for permanent removal due to a high risk of functional loss versus an extremely low security impact. These records are available at customer request.
For clarification: “Critical Updates” are patches that remedy a functional flaw in an operating system or product, but are not necessarily related to security. The category of “Security Updates” also carries a relative Severity, which can be Low, Moderate, Important, High, or Critical, hence the common confusion.
A note regarding Windows Home Edition: By design, this operating system is not designed for use in a corporate environment. While our management platform can attempt to manage Windows Home Edition updates, we cannot guarantee management of patching schedules for any Windows Home edition due to its structural limitations.
Our Windows patching schedule is as follows: Operating system patch and reboot windows for servers occur every Wednesday from 3:00am to 6:00am. Workstation patch and reboot windows occur daily, also from 3:00am to 6:00am. If a patch install begins but does not finalize within this window, no restart will be performed; the restart will be deferred to the next reboot window. In all cases, patch jobs occur only when there are missing updates that must be installed, and reboots only occur if required by the operating system.
In an effort to maximize update delivery effectiveness, our ManageSuite agents will also attempt silent, out-of-schedule update installs against desktops and laptops where possible, frequently within ten minutes after an out-of-date system comes online. Our Responsive Endpoint Patching system for out-of-date, out-of-schedule, client update delivery is fine-tuned to avoid business interruption from bandwidth consumption, slow computer response, or unwanted reboots. Systems updated in this manner will not restart automatically until their next scheduled reboot window. However, a user who shuts down or restarts a system that was silently patched during the day may see those updates finishing their installation at unexpected times. This Responsive Endpoint Patching system is NOT performed on servers.
It is a common misconception that out-of-date systems will immediately start patching the next time they are turned on; unfortunately, this is not always a practical promise to keep. Our Responsive Endpoint Patching system for out-of-date, out-of-schedule, silent update delivery is fine-tuned to avoid business interruption from bandwidth consumption, slow computer response, or unwanted reboots. We are always refining this process for greater patch delivery success. Clients for whom out-of-schedule update installs are business-disruptive should contact us to address this issue, or they can elect to opt-out of unscheduled updates altogether.
Some customers with second- or third-shift operations may choose to opt out of the Wednesday and Thursday installation schedule if this downtime would be disruptive. Instead, we offer the option of a conservative patch schedule where both server and workstation OS updates are installed every Sunday morning, also from 3:00am to 6:00am.
If a customer has one or more servers or workstations that either cannot be rebooted during the scheduled patch reboot window or simply cannot be patched at all, we can make per-system exclusions on an as-needed basis.
Our BRS units are owned and maintained by our staff; we schedule their update install and reboot windows independently of the customer’s patch schedule for better backup availability, low bandwidth impact, and reliable operation. Your environment will not be impacted by the regular maintenance of these units. Also, we strongly encourage clients with Customer-Owned Backup and Recovery Appliances (COBRAs) to allow us to configure a similar schedule for those servers, as the overnight replication of backup data might be interrupted by a standard 3am to 6am reboot schedule.
Our RMM tool effectively replaces your WSUS server, so related Group Policies and WSUS services in your environment will be disabled as a part of this service. Instead, we may opt to create one or more patch caches in your environment that effectively serve a similar function but do so more efficiently and with less network and disk space impact. Typical patch caches may occupy as much as 20 GB of space, but can be reset and cleared at any time if needed.
If your environment is missing a large number of patches at the time this service begins, it may take multiple weeks to fully update all resources. Additionally, all workstations and servers are required to be up-to-date with the most recent relevant OS Service Pack in order to be eligible to receive updates. Our initial audit will reveal if additional service is necessary to bring any endpoints into Service Pack compliance.
When reviewing patches installed by our RMM tool within your environment, please use the Programs and Features control panel and then select View Installed Updates within that. The RMM agent automatically detects and corrects problems with the Windows Update Agent, and those procedures may also reset the update history displayed through Windows Update. Therefore, the Windows Update screen may not accurately reflect the true update history of the resource. The list of installed updates in Programs and Features, on the other hand, will always be accurate.
Windows 10 Feature Update Delivery via ManageSuite Services
Windows 10 is part of Microsoft’s OS-as-a-service model, with regular, scheduled releases of new Windows 10 versions. This supplants the older practice of building service packs that augment the operating system instead of replacing it. The process of updating one version of Windows 10 to a newer version is called a “Feature Update”. Feature Updates are a single-action operating system upgrade and migration, which preserves (where possible) software installations, user profiles, and system configuration settings.
Microsoft typically assigns a support lifetime of 18 months for each Home and Pro editions of Windows 10, and 30 months for Education and Enterprise editions, though exceptions do occur. A full breakdown of the Windows 10 lifecycle is available here:
Unsupported versions of Windows 10 are not guaranteed to receive security updates from Microsoft after the end of their support lifetime. Therefore, in order for any system running Windows 10 to remain secure, it must receive regular Feature Updates in order to keep the installed operating system within its support lifetime.
Suite3 is currently delivering Windows 10 version 20H2 (build 19042).
This version is supported until May 10, 2022 for Home and Professional licenses, and May 9, 2023 for Enterprise and Education licenses. We’ve selected this version at this time due its combination of broad support among most Line of Business applications, relative support lifetime compared to other currently supported versions, and general reliability improvements.
Delivery and Installation
The download for a Feature Update is considerable, usually around 5 GB of data. Our automated scripts are able to perform these downloads using multiple methods to minimize bandwidth impact to the user and client. Once downloaded, the Feature Update procedure can be lengthy, requiring multiple reboots between 30 minutes and 2 hours of install time (depending on the age and speed of the system). Because of these factors, we install Feature Updates during our clients’ overnight patch delivery windows in order to avoid user interruption. Users whose systems are ready for a Feature Update install may see a pop-up message advising them to leave their computers turned-on, plugged-in, and internet-connected so the Feature Update may run. These pop-ups will continue to occur until a Feature Update attempt is initiated.
Suite3’s automated systems will not schedule more than 10 systems for Feature Updates per client per night, to avoid a high business impact in case problems do occur. We’re not able to publish in advance a list of systems that will be upgraded on any given night due to the highly variable availability of systems online during the overnight upgrade window. We’re also able to perform additional Feature Updates on-demand by request, just send a request to our service desk and we’ll make that happen.
The vast majority of Feature Update installs are successful and mostly transparent. After a workstation is updated, it’s very common for the very first login after the update to take a little longer (up to 5 minutes). Additionally, an updated system may ask for one more restart as various drivers are updated automatically. Following that, logins and behavior should perform as usual; please tell your users to be patient!
Application problems after a Windows 10 upgrade have been very rare in our testing, but they can occur. The most common fix for problematic applications is a Repair Install of the affected application (such as Microsoft Office). Additionally, some more esoteric Line of Business applications don’t always support the latest versions of Windows 10. As always, give us a call or submit a service request if you encounter issues you can’t resolve after a Windows 10 Feature Update. Also, if you’re aware of any compatibility issues among applications critical to your business that conflict with any particular Windows 10 version, let us know and we’ll work with you to formulate a solution.
Non-OS (Third Party) Updates
Third party patching is essential for any secure environment, as many applications like Adobe Reader, Java, and Firefox release updates that remedy security vulnerabilities and improve functionality. Our RMM platform also performs third party application patching during and after normal business hours. This is done through passive daytime scripting and also powering on the workstations with Wake-On-LAN calls where necessary. No endpoint reboots are performed in conjunction with third party application patching.
Updates of third party software will only update applications within their major version; whole number major version upgrades are generally not performed for compatibility reasons. For example, Java 7 Update 41 may be updated to Update 151 automatically, but will NOT be upgraded to Java 8. Exceptions to these rules include Flash and Firefox, whose whole-number versions are now considered minor revisions.
Updates for third-party applications are generally released into production within 1-2 days of their general release. Suite3 does not internally review these updates for functionality with line-of-business applications. This is due to the highly irregular release schedules of these updates, the difficulty of simulating the various ways in which third-party applications might be integrated into line-of-business applications, and the infrequent occurrence of business continuity issues due to bugs in third-party updates. Since these applications are not a part of the operating system or issued by the same vendor, loss of business due to misbehaving third-party applications is exceedingly rare.
The following is a list of third party applications that we update by default:
- Adobe Acrobat Std/Pro
- Adobe Air
- Adobe Reader
- Amazon Corretto
- Apple iTunes
- Apple Safari
- Google Chrome
- Mozilla Firefox (Including ESR versions)
- Mozilla Thunderbird
- Oracle/Sun Java*
- PDF Creator
- WebEx Connect
* Updates for Oracle Java 8 are no longer publicly available for businesses. See our blog article on the subject for more information.
Please do not hesitate to contact us with any questions, comments, or concerns.
Last Revised: June 3, 2021