Tuesday, August 15, 2017

Warning "Snapshot provider error 0xe00008526. Backup Exec could not locate a VSS software or hardware snapshot provider for this job."

Problem

Backup Exec unable to locate snapshot providers installed on the computer.

Error Message

V-79-57344-34086 - AOFO: Initalization failure on: "C:". AOFO used: Microsoft VSS. Snapshot provider error 0xe0008526: Backup Exec could not locate a VSS software or hardware snapshot provider for this job. Select a valid VSS snapshot provider for the target computer.


Application Log :
Log Name: Application
Source: VSS
Event ID: 7001
Computer:
 
Description:
VssAdmin: Unable to create a shadow copy: The volume shadow copy provider is not registered in the system.
Command-line: 'C:\Windows\system32\vssadmin.exe Create Shadow /AutoRetry=15
C: Drive properties window shows error about VSS Provider:
1. Go to Windows Explorer
2. Right click on C: drive
3. Click " Shadow Copies" tab.
4. Check for this error:
Initialization Failed:
Error: 0x80042304 : The volume shadow copy provider is not registered in the system.

Cause

The snapshot provider selected for the backup job is not installed on the server being backed up.

Solution

1.  Check that snapshot providers are installed on the system being backed up.
  • Open a command prompt.
  • Type "vssadmin list providers"
  • If nothing is returned, then there is most likely an issue with VSS that will most likely need assistance from Microsoft to resolve
    • You can also verify that VSS is properly registered by checking the following registry key:
      •  HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\VSS\Providers\{b5946137-7b9f-4925-af80-51abd60b20d5}
  • If writers not listed or show they are in an error state, reboot the server and run the command again
 


2.  Change the backup job to use a specific snapshot provider.
  •  Open the backup job properties.
  •  Navigate to Advance Open File.
  •  Confirm the "Use advanced open file option" is checked.
  •  Select the appropriate snapshot provider - "Microsoft Volume Shadow Copy Service" with "System" selected in the pull down menu is typically best:
  
 

Hyper-V Virtual Machine Error 0x80070569

If you find you are getting a similar error 0x80070569 on Hyper-V 2012, you might be experiencing something similar to this.
Try restarting the Hyper-V Virtual Machine Management Service.


I wasn't sure if I could restart this service while I had Virtual Machines running, but since it was on my home lab environment, I thought I'd give it a shot. The service restarted with no issues on any of my virtual machines. What's more, when I tried to create a virtual machine, it worked, no error. I'm not sure why this happens or why restarting the service fixes it as it seems by the error to be a permissions issue but as a work around it seems to do the trick.

Since installing Backup Exec FP3 Hyper-V backup jobs leave a checkpoint against the virtual machine

Problem

Before installing Backup Exec 15 Feature Pack 3 (FP3) , a Hyper-V backup job created a checkpoint (usually visible in the Hyper-V manager). This checkpoint results in the creation of  an AVHDX file against each VHDX file inside the storage containing the virtual machine (VM). However at the end of the backup job the checkpoint and associated AVHDX files are merged and removed/deleted.

Since installing FP3, it can be seen that Hyper-V backups now create a checkpoint (and associated AVHDX files), however even when the backup job finishes, a checkpoint now remains linked to the VM although the checkpoint will be updated during subsequent backup processes.

NOTE: Backup Exec 15 Feature Pack 4 (FP4) and later updates or later Backup Exec versions may also see such symptoms, although the default setting for FP4 and later depends on which Feature Pack (FP) was installed prior to installing FP4 and if it was installed over an FP that already includes the choice, what the current setting for Hyper-V processing was prior to updating.

Cause

In the newer versions of Hyper-V, Microsoft have introduced a potentially faster method of performing incremental backups and Backup Exec 15 FP3 implements this new method as a server default (for Hyper-V 2012 and later installations) . This means all Hyper-V jobs (not just newly created jobs) on a given Backup Exec server will use the new settings immediately after FP3 is installed.

The basic concept of the new method is:

    •    A child checkpoint (snapshot) is created to collect the changes.
    •    On incremental backup, a new child checkpoint is created for current changes and then the parent checkpoint is backed up.
    •    When the backup is complete, the child and parent checkpoint is merged leaving only the new combined checkpoint.
    •    This combined checkpoint remains on the system and is used as the new parent for the next backup.
    •    A full backup will backup the current complete state of the VM and not just the parent checkpoint


Extra information:

Within Backup Exec this new method is known as the Faster processing method for Incremental Hyper-V backups with the original method being called the Standard processing method

Backup Exec has supported this new method since Backup Exec 2014 was released however Backup Exec 15 FP3 enabled it as the default method where prior to this, to enable it required the use of Powershell Commands (scripts) which are no longer needed against the FP3 (or later) updates.

FP4 (and later) set the default Hyper-V processing Standard when installing over FP2 and earlier update levels, but honors the current setting when installing over FP3 (or later) update levels. 

The backup job log will clearly show which method was used to perform the backup operation.

With the faster method, a checkpoint is created and remains in place against the VM even if the backup strategy is to only run full backups.

It is possible to revert to the Standard method (see information in the Solution section below), however, it should be noted that if reverting to the Standard method where at least one backup job with the Faster method has already run, then Powershell Commands will be needed to remove the Backup Checkpoints. Typically Hyper-V 2012 R2 installations will not remove the Backup Checkpoints when changing method, but Hyper-V 2012 installations usually will remove the Backup Snapshots. This is because Microsoft changed the behaviour (and the terminology) between these two Operating System versions.

The AVHDX file is often referred to as a differencing disk

The very first time a Faster method backup is performed against a VM, it will result in the Hyper-V-VMMS log on the host containing events 15070 and 10150,  reporting  errors of being unable to remove  a checkpoint. These errors can be ignored unless they happen during every backup operation.

A restore from a backup set created using one method will still work if the server settings have been changed to the other method.

If more than 1 backup checkpoint is present (when a backup job is not running) or more than 2 backup checkpoints present (when a backup job is running) then an issue has occurred that has blocked the removal of a checkpoint. As Backup Exec is calling Microsoft processes it is likely the cause is within the Operating System and not Backup Exec (unless you can see service stability issues against Backup Exec at the time of the backup.)
 

Solution

Backup Admins managing Hyper-V environments should choose between electing to continue using the default Faster method (in which case no further action is necessary) or reverting back to the Standard method. To revert to the Standard method perform the following steps:
  • In the Backup Exec console click the Backup Exec button (yellow button top left) and select Configuration and Settings —> Backup Exec Settings.
  • In the left sidebar of the Settings screen, click on Virtual Machines
  • Change the radio button in the Hyper-V, Incremental Backup Settings to “Use the standard processing method” and click OK
  • For Hyper-V Windows 2012 R2 installation then go to the Hyper-V host and open Powershell
To merge and remove all snapshots use the command
Get-VMSnapshot -VMName 'VM' -ComputerName 'HOST' | Remove-VMSnapshot
although if there are manual checkpoints against the VM that need retaining the to remove just the Backup Checkpoints use the command
Get-VMSnapshot -VMName 'VM' -ComputerName 'HOST' -SnapshotType Recovery | Remove-VMSnapshot

Where VM represents the actual name of your virtual machines as seen in the Hyper-V Manager on the host and HOST represents the name of the Hyper-V host

Note: these commands are case sensitive and can also be used to remove backup checkpoints that due to operating system errors have not been removed correctly. However, customers with these types of issue  should also review the Hyper-V-VMMS logs to understand what may have caused the condition.

Saturday, August 12, 2017

DFSR Event 5012 (DFS Replication)

Summary

DFS Replication failed to communicate with a replication group member because the member didn’t have the same configuration information. DFS Replication will retry this operation, which might resolve after configuration changes replicate through Active Directory Domain Services (AD DS).

Causes

DFS Replication could not establish a connection with the member. This error can occur for the following reasons:
  • An administrator recently made a configuration change or created a new replication group. If two replication group members obtain their configuration data from different domain controllers, they may have mismatched configuration data due to AD DS replication latencies.
  • An administrator altered AD DS permissions.


Resolutions


Wait for configuration changes to replicate with AD DS

DFS Replication will retry this operation, which usually resolves automatically after configuration changes replicate through AD DS.
To poll immediately for configuration changes, open a command prompt window and then type the following command once for each member of the replication group, where is the domain the server is located in and is the server name:
Dfsrdiag PollAD /Member:\
If this problem persists, verify that AD DS replication is working and that the service is able to reach a domain controller. If the problem persists for 8 hours, the DFS Replication service will disable the connection and log DFS Replication Event 5016.

Test AD DS connectivity

To test DFS Replication’s ability to communicate with AD DS, open a command prompt window and then type the following command: Dfsrdiag pollad /verbose. The command should report Operation Succeeded. If not, there is a problem communicating with AD DS.


Verification

To confirm that the issue is resolved, look for DFS Replication Event 5004 in the Event Viewer.crusoe
To manually verify the replication topology, see Repair a Disconnected Topology (http://go.microsoft.com/fwlink/?LinkId=187103 )

Friday, August 4, 2017

Windows Server 2012 How To Create System Restore Point Using Windows Server Backup Feature

In this article, let us see how to enable and use the backup features of Windows Server 2012/2016. Let us go step by step as usual.
1. Go To Server Manager.
clip_image002
2. Click on Next.
clip_image004
3. Select Role-based or Feature based installation and click Next.
clip_image006
4. Select the Server and Click Next.
clip_image008
5. The default roles will be selected. Don’t bother about them and click next.
clip_image010
6. On the Features Wizard, Select the Windows Server Backup Feature and click Install.
clip_image012
7. Once, the installation completes, do a restart of the server and execute the below exe from the Run.
clip_image013
8. The following wizard will appear.
clip_image015
9. On the Right Panel, you will find various options like Backup Schedule, Backup Once, Recover etc., they are self-explanatory as the name explains. Now, let us go with a Backup Once option. For this wizard, I request you to follow the screen shots as there is no explanation required for them.
clip_image017
clip_image019
clip_image021
clip_image023
clip_image025
clip_image027
Now, After the Backup, the System Restore Point will be Created.
Issues faced while creating a Backup.
1. Backup failed to complete. A Volume Shadow Copy Service Operation failed. Please check "VSS" and "SPP" application event logs for more information. Detailed error: The operation ended before completion.
Looking into the EventVWR, the detailed exception was like
The backup operation that started at '‎2013‎-‎10‎-‎08T07:37:24.827000000Z' has failed because the Volume Shadow Copy Service operation to create a shadow copy of the volumes being backed up failed with following error code '0x807800A1'. Please review the event details for a solution, and then rerun the backup operation once the issue is resolved.
To resolve this, I had a look into the below link and got out.
2. System Writer not found.
This is one of the important exception, which I got struck for 2 days and after a long time, I found a useful link about this. There are already a lot of discussions are going on around this.
And moreover, a recent patch for this from Microsoft can be downloaded from here.
Install the hotfix and do a restart. System writer will be listed on your system. To confirm, run the command prompt as administrator and type “vssadmin list writers”.
3. Make sure that, the VSS Service is running.
Probably, that’s all about this. Please do let me know, if you feel that something has been missed.