AWS EC2 Cost and Usage report

AWS provides a nice service called Cost Explorer to generate different AWS expense reports based on different filters and time range. It has a very easy to use interface to visualize cost and usage data through graphs as well as raw CSV data. You may refer AWS documentation to know how to use Cost Explorer in your environment.

However, I am going to talk about the alternate way of gathering AWS cost and usage data using AWS Cost Explorer CostAndUsage APIs. Let’s assume you need to automate AWS cost and usage data collection and pass it to other tools/systems for further processing – may be to some database or to PowerBi. I recently, worked on a project, where we wanted to generate cost and usage raw data using powershell and automate it in a schedule. You still have ways of dumping Billing data to S3 Bucket or process it through Athena and visualizing using Quicksight. Definitely, those are nice services provided by AWS and makes our life easy.But, I trusted powershell and AWS APIs for obvious reasons 🙂

Here I will be using AWS Tools for powershell and make calls to cost and usage API – Get-CECostAndUsage . Refer here to get detailed documentation on the SDK. I am generating cost and usage report for EC2 service. However, you can use it for any other service too.

I am defining Dimension as “Service”. If you have more than one accounts , you can use “Linked Account” as Dimension. Here is the script snippet to set “Service” as key to Dimension JSON property and “Amazon Elastic Compute Cloud – Compute” as value.

$dimension = New-Object Amazon.CostExplorer.Model.DimensionValues
$dimension.Key = "SERVICE"
$dimension.Values ="Amazon Elastic Compute Cloud - Compute"

Now we will include it in the Filter. Again, refer to AWS documentation for more information.So, here, I am defining the Filter to be part of the SDK call.

$Filter = New-Object Amazon.CostExplorer.Model.Expression
$Filter.Dimensions = $dimension

As we are generating report for all EC2 instances in my account, I will use EC2 instance “Name” Tag as the Grouping type. This will generate the cost and usage output group by the EC2 instance Name.

$groupInfo = New-Object Amazon.CostExplorer.Model.GroupDefinition
$groupInfo.Type = "TAG"
$groupInfo.Key = "Name"

Now, for Metric data, I am using both “BlendedCost” and ” UsageQuantity”. Both are required as I am trying to capture both cost and usage hours.

$metric = @("BlendedCost","UsageQuantity")

We also need to define the time interval for the time period, I will be generating this report. Here I am using $firstDay and $lastDay to define the dates for the report period. Example, “2019-08-01” and “2019-09-01”.

$interval = New-Object Amazon.CostExplorer.Model.DateInterval
$interval.Start = $firstDay
$interval.End = $lastDay

Finally, we have all required data to make the API call. Let’s make the call now :

$costUsage = Get-CECostAndUsage -TimePeriod $interval -Granularity DAILY -Metric $metric -Filter $Filter -GroupBy $groupInfo

Now we have the data in $costUsage data.We just need to format it in a proper format , so that, the raw data is still readable and can be ingested to any system without making any changes.

Here I am giving the complete script to generate CSV report. Let me know if it helps or you have any issue running it.

Param(
[Parameter(Mandatory=$false)]
[String]$firstDay,
[Parameter(Mandatory=$false)]
[String]$lastDay
)

#Print cost details for the Account for previous month if no dates provided
if([String]::IsNullOrEmpty($firstDay)){
    $firstDay = Get-Date -Year (Get-Date).Year -Month (Get-Date).AddMonths(-1).Month -Day 1 -UFormat "%Y-%m-%d"
}
if([String]::IsNullOrEmpty($lastDay)){
    $lastDay = Get-Date -Year (Get-Date).Year -Month (Get-Date).AddMonths(-1).Month -Day ([DateTime]::DaysInMonth((Get-Date).Year, (Get-Date).Month)) -UFormat "%Y-%m-%d"
}
$currentDir = $(Get-Location).Path
$oFile = "$($currentDir)\aws_ec2_billing_usage_data.csv"
$tFile = "$($currentDir)\aws_ec2_billing_usage_data_temp.csv"
if(Test-Path $oFile){
    Remove-Item $oFile -Force
}
if(Test-Path $tFile){
    Remove-Item $tFile -Force
}
"Name,Date,UsageHours,Cost" | Out-File $tFile -Append -Encoding ASCII
$interval = New-Object Amazon.CostExplorer.Model.DateInterval
$interval.Start = $firstDay
$interval.End = $lastDay

$dimension = New-Object Amazon.CostExplorer.Model.DimensionValues
$dimension.Key = "SERVICE"
$dimension.Values ="Amazon Elastic Compute Cloud - Compute"
$Filter = New-Object Amazon.CostExplorer.Model.Expression
$Filter.Dimensions = $dimension
$groupInfo = New-Object Amazon.CostExplorer.Model.GroupDefinition
$groupInfo.Type = "TAG"
$groupInfo.Key = "Name"

$metric = @("BlendedCost","UsageQuantity")

$costUsage = Get-CECostAndUsage -TimePeriod $interval -Granularity DAILY -Metric $metric -Filter $Filter -GroupBy $groupInfo
ForEach($c in $costUsage.ResultsByTime){
    $sTime = $cost = $instanceName = ""
    $sTime = $c.TimePeriod.Start
    ForEach($grp in $c.Groups){
        $instanceName = $grp.Keys.Split("$")[1]
        if([String]::IsNullOrEmpty($instanceName)){$instanceName = "No Name Tag"}
        $cost = $grp.Metrics["BlendedCost"].Amount
        $usageHours = $grp.Metrics["UsageQuantity"].Amount
        "$instanceName,$sTime,$usageHours,$cost" | Out-File $tFile -Append -Encoding ASCII
    }
}

Import-Csv $tFile | Sort-Object 'Name','Date' | Export-Csv -Path $oFile -NoTypeInformation
if(Test-Path $tFile){
    Remove-Item $tFile -Force
}