The app behaves like a model for a coal power plant which has:
a. a burner unit where coal is ingested and combusted to produce heat which then is used for electricity and
b. a central hub which collates data from many such burner units to get a wholesome picture of the emissions situation in a given area/grid network.
Based on the above, this app has two parts to it:
the emitter service (NRChallenge_Services) : I didn't find an open source dataset that fitted my data needs so I created my own data store and API ;). This service 'emits' random CO2, SO2 and NOX values along with other important indicators of the health of a power plant setup.
the controller service (NRChallenge_Controller): This provides for two needs. It acts as a consumer of the values from the emitter service and does its own calculations which it makes available on a browser app. This could be where a not-very-tech-savvy person views the daily activity of the plant(s).
Both services are built using the Django framework and the default sqlite database as a data store. The emitter app is really an API created using Django Rest Framework. The controller gets its data from the emitter service through an AWS Lambda function.
Finally, all these are monitored for various events and metrics by NEW RELIC!
Science and observation
The way the controller app runs is that it requires a Django background task to run. This background task runs every minute to get fresh data from the emitter app.
Deploying the background task seemed to be cumbersome to deploy. Without that background task, the UI itself is of no apparent value. Hence attaching a screencast of the local build of the app here complete with the background task.
The emitter app is here : Emitter app
/co2 to see data for about 4 hours on 2021-02-28.
It is easy to see the relation between CO2 emissions and the heat rate. They are inversely proportional. Subsequently, CO2 emissions and plant efficiency are directly proportional. Fixing efficiency issues could not just reduce the CO2 emitted but also save the company money in carbon taxes and fossil fuel costs, not to mention storage infrastructure.
The values in the screenshot above are cumulative values i.e., the displayed values are a culmination of all emission, rate etc values till the present time.
The Datalyzer could be used in a case of multi-location plant monitoring, with the guid values representing various location IDs for example.
Coal and gas power plants that produce 65% of the electricity are the largest producers of carbon dioxide and other earth-warming gases (called Greenhouse gases or GHG ) in the US.
The app consists of two parts: the emitter which produces near-real-life emission values each minute which are then consumed by the controller which produces them on a UI after more
calculations. This could then be viewed by people managing the infrastructure of the plant and take reasonable actions.
The decision to split these into two apps was deliberate because that might well be the case in real plant setups; emission towers where the burning happens in location 1 and a more centralized hub to collate results from many such towers.
The metrics considered are :
- CO2 Emissions: Should be low
- SO2 Emissions: Should be low
- NOX Emissions: Should be low
- Heat rate: Should be low
- Plant efficiency: Should be high. Depends on the heat rate and is inversely proportional
- CO2 Reserve: This is related with the carbon capture and sequestration process which dumps collected carbon dioxide from the combustion process into storage units. These units have a limit to ensure the gas doesn't have adverse ecological effects.
- Expected reduction targets: In order to compare current values, we need to have a target value of reduction of emissions to be in line to meet the emissions goal for 2025 and eventually 2050.
Note: The values considered for the targets are mock values. The random emissions of CO2, SO2 and NOX are close-to-reality values, however. The information sources are enlisted at the end of this post. Also, I have assumed values for the US because it's the largest historic producer of GHG. This could be applied to any country's setup.
The motivation for this app was the recent USD $100m prize Elon Musk announced for a revolutionary carbon capture technology.
- Observability is key: This app is based on the real-time monitoring capability that New Relic champions. “If we want to get serious about curbing carbon emissions, we need to measure them with precision. Looking at data at the hourly level, instead of monthly or yearly as is typically done, is a much more targeted way to accurately assess and control emissions.” says PhD student Jacques de Chalendar at Stanford, 2019. Observability of existing power plants' emission outputs would be a game changer for the global climate crisis. Only if we know what is happening currently could we make changes to improve that in the direction of zero emissions by 2050 (aka the Paris Agreement)
- Time-based : New Relic has the capability to display a lot of data into insights that make sense to anyone who can read graphs
- Ease of use : The extremely-easy integration of NR into an existing app and close-to-nothing setup time were total winners for me. DevOps must be easy and NR lives up to that belief.
- APM: The Application Performance Monitoring is the heart of the app. It is used to monitor both the emitter and the controller apps.
- Custom metrics: CO2/SO2/NOX emissions, heat rates, plant efficiencies, CO2 reserve values and the expected reduction targets are all monitored using the custom metric feature to produce dashboard graphs and sensible insights
- Dashboards: Used on both the emitter and controller apps
- Alerts and notification channels, and subsequent Incidents: The emission values, heat rates and CO2 reserve have alert policies set up which when violated send emails to the set notification channel
- AWS Lambda infrastructure setup: The emitter values are made available to the controller via a Lambda integration which is NR monitors. The emitter is expected to output values continuously as the 'coal' is combusted which would an auto-scaling capability hence AWS Lambda. Alerts have been applied to the integration as well to mark issues with the infrastructure.
- Service maps: In a real-world monitoring scenario, there would be a network of power plants with their own emitter and controller apps which would report to a central monitor hub. This app has this set up albeit for a single power plant setup
- Distributed tracing: In continuation from service maps, distributed tracing can further simplify the incidence discovery process
- Datalyzer: For a real-world setup of power plants, a grid network might consist of various power plants in multiple locations. The Datalyzer app could make this location-wise power plant visualization easier.
- (not applied but desirable): The SLA tool could be used to view weekly reports of improvements or otherwise but doesn't have a custom metric reporter yet. Carbon capture and carbon reduction plans require to make changes to existing plant infrastructure. Being able to view differences before and after infrastructure rejig would be a great addition.
- Read about the Paris Agreement here.