10/09/2013: Here's a script I've used in various forms since around 2007: A basic rolling Exchange Hubtransport Queue Depth monitor.I've found over time, that regardless of SCOM or other active monitoring in place for your Exchange infrastructure, you can often see problems show up early and first in queuing behavior on your HubTransport servers:
- Experiencing mail server storage latency issues? You'll probably see increasing patterns of queuing in the MapiDelivery queues to the problem stores, as you move into peak business primetime hours and increasing mail load delivers into the hubs, but can't be delivered into the mailbox server transaction logs and stores, fast enough to keep pace with the mail flow.
- Experiencing cpu-saturation, or other capacity issues on your hubs, or Hub-CAS hybrid systems? You'll probably see the footprint on the CPU-dependant AV-scanning processes (you ARE running antivirus scanning on your hubs, correct? :D), which will lead to mail queuing in the Submissions queues.
--in my case, I most recently saw this as a product of CAS Exchange Web Services load -- from Lync -- soaking up more CPU than the systems had been provisioned for, in 2007, well before the Lync infrastructure had been tacked onto the Exchange infrastructure -- looong story behind that one, that predated my arrival at the firm. :P))
So in a nutshell, I've found it a pretty good backup-monitoring option, to run a self-refreshing console display on the current queue status of all hubs in a given Exchange revision.
Toward that end, here's the current revision of my tool for the purpose, 'HT-queues-info-loop-E710.ps1', which does nothing more than output the current queue status for all hubs at a given Exchange revision level, and refresh the display every 15 seconds.
The functional outline for the script is the following:
Note: The script is designed to accommodate use in either Exchange 2007 or Exchange 2010 (under both Powershell1 & Powershell2), and dynamically detect the current hosting Exchange version it is running on.
- The script dynamically tests for and loads the most-current Exchange Management Shell version available on the system.
- It then pulls a list of all HubTransport servers on the matching Exchange revision, into an array.
- And loops that array every 15 seconds, pulling and displaying the current active queues -- by default I tend to run it displaying only non-zero queues (configured via the $bNonZero variable).
HT-queues-info-loop-E710.ps1 - Reports HubTrans queue depth on a loop every 15secs - hybrid Ex2007/Ex2010 version
Language: Powershell
Target: EMS or Powershell
# HT-queues-info-loop-E710.ps1
# .SYNOPSIS
#HT-queues-info-loop-E710.ps1 - Reports HubTrans queue depth on a loop every 15secs - hybrid Ex2007/Ex2010 version
#
# .NOTES
#Written By: Todd Kadrie
#Website: http://tinstoys.blogspot.com
#Twitter: http://twitter.com/tostka
#
# NOTE: THIS SAMPLE CODE HAS BEEN LINE-WRAPPED AT PIPE ( | ) COMMANDS
# AND MID-LINE SEMICOLONS ( ; ), TO IMPROVE READABILITY ON THE WEB SITE.
# The code will work without issues with these line-wraps,
# but I traditionally have these sections un-wrapped for production use.
#
#Change Log
# rev: 8:40 AM 10/9/2013 initial public version
#
# Script to display queue information of Hub Transport Servers
# Dynamicly selects Exchange Hub Transport Servers from Exchange Organization
# Selects server's location by Site
# Output queue information
#
# .DESCRIPTION
#HT-queues-info-loop-E710.ps1 - Reports HubTrans queue depth on a loop every 15secs
#
# .INPUTS
#None. Does not accepted piped input.
#
# .OUTPUTS
#Outputs to console.
#
# .EXAMPLE
#.\HT-queues-info-loop-E710.ps1
#------------- Declaration and Initialization of Global Variables -------------
## immed after param, setup to stop executing the script on the first error
trap { break; }
# Set pause period (secs) & scale up to ticks
$sleeptime = 15 ; $sleeptime *= 1000
# Set to $TRUE to display only NONZERO queues. False, displays all queue depths (also overridden at EMSVers level below)
$bNonZero = $True
# debugging flag
#$bDebug = $TRUE
$bDebug = $FALSE
If ($bDebug -eq $TRUE) {
write-host ((get-date).ToString("HH:mm:ss") + ": *** DEBUGGING MODE ***")
}
# quote & singlequote characters
$sQot = [char]34 ;
$sQotS = [char]39 ;
#*================v FUNCTION LISTINGS v================
#*----------v Function EMSLoadLatest v----------
# Checks local machine for registred E20[13|10|07] EMS, and then loads the newest one found
# Returns the string 2013|2010|2007 for reuse for version-specific code. Note, the E13 code requires further testing
# # vers: 2:05 PM 7/19/2013 typo fix in 2013 code
# # vers: 1:46 PM 7/19/2013
# .NOTES
#Written By: Todd Kadrie
#Website: http://tinstoys.blogspot.com
#Twitter: http://twitter.com/tostka
#
function EMSLoadLatest {
# compare registered vs loaded plugins;
$SnapsReg=Get-PSSnapin -Registered ;
$SnapsLoad=Get-PSSnapin ;
# check/load E2013, E2010, or E2007, stop at newest (servers wouldn't be running multi-versions)
if (($SnapsReg |
where {$_.Name -eq "Microsoft.Exchange.Management.PowerShell.E2013"}))
{
if (!($SnapsLoad |
where {$_.Name -eq "Microsoft.Exchange.Management.PowerShell.E2013"}))
{
Add-PSSnapin Microsoft.Exchange.Management.PowerShell.E2013 -ErrorAction SilentlyContinue ;
return "2013" ;
} else {
return "2013" ;
}
} elseif (($SnapsReg |
where {$_.Name -eq "Microsoft.Exchange.Management.PowerShell.E2010"}))
{
if (!($SnapsLoad |
where {$_.Name -eq "Microsoft.Exchange.Management.PowerShell.E2010"}))
{
Add-PSSnapin Microsoft.Exchange.Management.PowerShell.E2010 -ErrorAction SilentlyContinue ;
return "2010" ;
} else {
return "2010" ;
}
} elseif (($SnapsReg |
where {$_.Name -eq "Microsoft.Exchange.Management.PowerShell.Admin"}))
{
if (!($SnapsLoad |
where {$_.Name -eq "Microsoft.Exchange.Management.PowerShell.Admin"}))
{
Add-PSSnapin Microsoft.Exchange.Management.PowerShell.Admin -ErrorAction SilentlyContinue ;
return "2007" ;
} else {
return "2007" ;
}
}
}
#*----------^END Function EMSLoadLatest ^----------
#*================^ END FUNCTION LISTINGS ^================
#--------------------------- Invocation of SUB_MAIN ---------------------------
# invoke EMS module load
$EMSVers = EMSLoadLatest # return's a string ["2013"|"2010"|"2007"];
# test $EMSVers for approp code in multi-version scripts (2007|2010|2013)
write-host ((get-date).ToString("HH:mm:ss") + ": `$EMSVers: " + $EMSVers)
$ComputerName = ($env:COMPUTERNAME)
# Dynamicly set variables for each site's hub transport servers, and sort by server name
# sample syntax to gather site names:
# [PS] C:\Windows\System32>Get-ExchangeServer | Where { $_.isHubTransportServer -eq $true } | sort name
# returns two sites as dns, "domain.com/Configuration/Sites/Site1" & "domain.com/Configuration/Sites/DisasterRecoveryCenter"
# switch on running EMSvers, and configure options to suit
switch ($EMSVers){
"2007" {
if ($bDebug) {write-host "2007"}
# if we're on E2007, force nonzero - too many hubs/queues displayed otherwise
$bNonZero = $TRUE
$hubs = Get-ExchangeServer |
where {$_.isHubTransportServer -eq $true -AND $_.admindisplayversion.major -eq 8 } |
sort name
}
"2010" {
if ($bDebug) {write-host "2010"}
$bNonZero = $TRUE
#$E10Hub = Get-ExchangeServer |where {$_.isHubTransportServer -eq $true -AND $_.admindisplayversion.major -eq 14 } | sort name
$hubs = Get-ExchangeServer |
where {$_.isHubTransportServer -eq $true -AND $_.admindisplayversion.major -eq 14 } |
sort name
}
"2013" {
if ($bDebug) {write-host "2013"}
# Note: The E2013 code for this script is largely a placemarker and unimplemented; requires further revisions...
#$E13Hub = Get-ExchangeServer | where {$_.isHubTransportServer -eq $true -AND $_.admindisplayversion.major -eq 16 } | sort name
# 2:11 PM 10/7/2013 E2013 messes up the admindisplayversion - MS changed the variable type from Version to String! Requires a workaround to function.
# unimplemented yet.
}
} # switch block end
# run a for-based infinite loop
for (;;)
{
write-host " "
# echo time per pass
$sMsg = (get-date).toshortdatestring() + "-" + (get-date).toshorttimestring()
Write-Host $sMsg -foregroundcolor "gray"
# configure colors for queue displays
[console]::ForegroundColor = "Green"
[console]::ResetColor()
# run a list of all hub stats in $hubs site
foreach ( $hub in $hubs )
{
[console]::ForegroundColor = "Green"
write-host "----" $hub ":"
# samples of the hub queue queries being run below (formats the output to shrink the displayed footprint for full-time desktop monitoring loop display):
# prod Exch2010 version, note: using only single quotes
# get-exchangeserver us-hubcas1 -erroraction silentlycontinue | get-queue -erroraction silentlycontinue | where {$_.MessageCount -gt 0} | select @{Label='Identity';Expression={(($_.Identity.ToString())).(0,[System.Math]::Min(15, $_.Identity.Length)) + '...'}},@{Label='Type';Expression={(($_.DeliveryType.ToString())).substring(0,6) + '...'}},Status,@{Label='Count';Expression={$_.MessageCount}},@{Label='NxtHop';Expression={(($_.NextHopDomain.ToString())).substring(0,[System.Math]::Min(16, $_.NextHopDomain.Length)) + '...'}} | ft -auto | out-default
# prod Exch2007 version single quotes
# get-exchangeserver us-hubcas1 -erroraction silentlycontinue | get-queue -erroraction silentlycontinue | where {$_.MessageCount -gt 0} | select @{Name='Identity';Expression={(($_.Identity.ToString())).(0,[System.Math]::Min(15, $_.Identity.Length)) + '...'}},@{Name='Type';Expression={(($_.DeliveryType.ToString())).substring(0,6) + '...'}},Status,@{Name='Count';Expression={$_.MessageCount}},@{Name='NxtHop';Expression={(($_.NextHopDomain.ToString())).substring(0,[System.Math]::Min(16, $_.NextHopDomain.Length)) + '...'}} | ft -auto
# begin building the queue query string, a piece at a time...
$sCmd = "get-exchangeserver " + $hub + " -erroraction silentlycontinue | get-queue -erroraction silentlycontinue"
if ($bNonZero) {
# filter/display only non-zero queues
$sCmd +=" | where {`$_.MessageCount -gt 0}"
}
# append approp SELECT block (defaulting to Exch2010 syntax, correcting for E2007 below)
$sCmd +=" | select @{Label='Type';Expression={((`$_.DeliveryType.ToString())).substring(0,6) + '...'}},Status,@{Label='Count';Expression={`$_.MessageCount}},@{Label='NxtHop';Expression={((`$_.NextHopDomain.ToString())).substring(0,[System.Math]::Min(16, `$_.NextHopDomain.Length)) + '...'}}"
# for 2007 Powershell Ver1, we need to replace LABEL custom field strings with NAME to avoid an error...
if (($EMSVers -eq "2007") -AND ($host.version.major -eq "1")) {
if ($bDebug) {write-host "doing Exch2007 Label:Name swap..."}
$sCmd=$sCmd.Replace('Label', 'Name')
}
# append the output formatting
$sCmd +=" | ft -auto"
if ($EMSVers -eq "2010"){
# 2010 needs added out-default to workaround error piping format-table:
$sCmd +=" | ft -auto | out-default"
}
if ($bDebug) {
# echo the current query syntax being run
write-host -foregroundcolor darkgray ((get-date).ToString("HH:mm:ss") + ": `$sCmd: " + $sCmd)
write-host -foregroundcolor darkgray ((get-date).ToString("HH:mm:ss") + ": Invoking `$sCmd...")
}
# pre-clear errors, invoke the command, and then test for and echo any errors...
$error.clear()
Invoke-Expression $sCmd -verbose
if($error.count -gt 0){
write-host -foregroundcolor red ((get-date).ToString("HH:mm:ss") + " ERROR: " + $error )
}
} # HUB for-loop end
write-host "======" -foregroundcolor "gray"
$sMsg = "pausing (" + ($sleeptime / 1000) + " secs)..."
Write-Host $sMsg -foregroundcolor "gray"
Start-Sleep -m $sleeptime
} # MAIN for-loop end
Typical output:
10/9/2013-4:36 PM
---- US-HUBCAS1 :
Type Status Count NxtHop
---- ------ ----- ------
SmtpRe... Active 81 site1...
SmtpRe... Active 36 site2...
---- US-hubcas2 :
Type Status Count NxtHop
---- ------ ----- ------
SmtpRe... Active 2 site2...
SmtpRe... Active 1 site1...
Undefi... Ready 1 Submission...
---- US-HUBCAS3 :
Type Status Count NxtHop
---- ------ ----- ------
SmtpRe... Active 1 site2...
SmtpRe... Active 5 site1...
SmtpRe... Active 1 hub version 14...
Undefi... Ready 3 Submission...
======
pausing (15 secs)...
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.