GitHub/school-of-sre
1
0
Fork 0
mirror of https://github.com/linkedin/school-of-sre synced 2026-01-07 17:18:03 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
0fb7d395b1fb1e8bd9ad4b8f22488d29248fa74a
school-of-sre/systems_design/availability/index.html
Kalyanasundaram Somasundaram 0fb7d395b1 Deployed 4eb864e with MkDocs version: 1.1.2
2020-11-17 13:49:14 +05:50

1161 lines
28 KiB
HTML
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<!doctype html>
<html lang="en" class="no-js">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<link rel="shortcut icon" href="../../assets/images/favicon.png">
<meta name="generator" content="mkdocs-1.1.2, mkdocs-material-6.1.5">
<title>Availability - SchoolOfSRE</title>
<link rel="stylesheet" href="../../assets/stylesheets/main.21aed14c.min.css">
<link rel="stylesheet" href="../../assets/stylesheets/palette.196e0c26.min.css">
<link href="https://fonts.gstatic.com" rel="preconnect" crossorigin>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto:300,400,400i,700%7CRoboto+Mono&display=fallback">
<style>body,input{font-family:"Roboto",-apple-system,BlinkMacSystemFont,Helvetica,Arial,sans-serif}code,kbd,pre{font-family:"Roboto Mono",SFMono-Regular,Consolas,Menlo,monospace}</style>
</head>
<body dir="ltr" data-md-color-scheme="" data-md-color-primary="none" data-md-color-accent="none">
<input class="md-toggle" data-md-toggle="drawer" type="checkbox" id="__drawer" autocomplete="off">
<input class="md-toggle" data-md-toggle="search" type="checkbox" id="__search" autocomplete="off">
<label class="md-overlay" for="__drawer"></label>
<div data-md-component="skip">
<a href="#ha-availability-common-nines" class="md-skip">
Skip to content
</a>
</div>
<div data-md-component="announce">
</div>
<header class="md-header" data-md-component="header">
<nav class="md-header-nav md-grid" aria-label="Header">
<a href="../.." title="SchoolOfSRE" class="md-header-nav__button md-logo" aria-label="SchoolOfSRE">
<img src="../../img/sos.png" alt="logo">
</a>
<label class="md-header-nav__button md-icon" for="__drawer">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M3 6h18v2H3V6m0 5h18v2H3v-2m0 5h18v2H3v-2z"/></svg>
</label>
<div class="md-header-nav__title" data-md-component="header-title">
<div class="md-header-nav__ellipsis">
<span class="md-header-nav__topic md-ellipsis">
SchoolOfSRE
</span>
<span class="md-header-nav__topic md-ellipsis">
Availability
</span>
</div>
</div>
</nav>
</header>
<div class="md-container" data-md-component="container">
<main class="md-main" data-md-component="main">
<div class="md-main__inner md-grid">
<div class="md-sidebar md-sidebar--primary" data-md-component="navigation">
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--primary" aria-label="Navigation" data-md-level="0">
<label class="md-nav__title" for="__drawer">
<a href="../.." title="SchoolOfSRE" class="md-nav__button md-logo" aria-label="SchoolOfSRE">
<img src="../../img/sos.png" alt="logo">
</a>
SchoolOfSRE
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../.." class="md-nav__link">
Home
</a>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle" data-md-toggle="nav-2" type="checkbox" id="nav-2" >
<label class="md-nav__link" for="nav-2">
Fundamentals Series
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" aria-label="Fundamentals Series" data-md-level="1">
<label class="md-nav__title" for="nav-2">
<span class="md-nav__icon md-icon"></span>
Fundamentals Series
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle" data-md-toggle="nav-2-1" type="checkbox" id="nav-2-1" >
<label class="md-nav__link" for="nav-2-1">
Git
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" aria-label="Git" data-md-level="2">
<label class="md-nav__title" for="nav-2-1">
<span class="md-nav__icon md-icon"></span>
Git
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../git/git-basics/" class="md-nav__link">
Git Basics
</a>
</li>
<li class="md-nav__item">
<a href="../../git/branches/" class="md-nav__link">
Working With Branches
</a>
</li>
<li class="md-nav__item">
<a href="../../git/github-hooks/" class="md-nav__link">
Github and Hooks
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle" data-md-toggle="nav-2-2" type="checkbox" id="nav-2-2" >
<label class="md-nav__link" for="nav-2-2">
Linux Networking
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" aria-label="Linux Networking" data-md-level="2">
<label class="md-nav__title" for="nav-2-2">
<span class="md-nav__icon md-icon"></span>
Linux Networking
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../linux_networking/intro/" class="md-nav__link">
Introduction
</a>
</li>
<li class="md-nav__item">
<a href="../../linux_networking/dns/" class="md-nav__link">
DNS
</a>
</li>
<li class="md-nav__item">
<a href="../../linux_networking/udp/" class="md-nav__link">
UDP
</a>
</li>
<li class="md-nav__item">
<a href="../../linux_networking/http/" class="md-nav__link">
HTTP
</a>
</li>
<li class="md-nav__item">
<a href="../../linux_networking/tcp/" class="md-nav__link">
TCP
</a>
</li>
<li class="md-nav__item">
<a href="../../linux_networking/ipr/" class="md-nav__link">
Routing and Conclusion
</a>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle" data-md-toggle="nav-3" type="checkbox" id="nav-3" >
<label class="md-nav__link" for="nav-3">
Python and Web
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" aria-label="Python and Web" data-md-level="1">
<label class="md-nav__title" for="nav-3">
<span class="md-nav__icon md-icon"></span>
Python and Web
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../python_web/intro/" class="md-nav__link">
Introduction
</a>
</li>
<li class="md-nav__item">
<a href="../../python_web/python-concepts/" class="md-nav__link">
Some Python Concepts
</a>
</li>
<li class="md-nav__item">
<a href="../../python_web/python-web-flask/" class="md-nav__link">
Python, Web and Flask
</a>
</li>
<li class="md-nav__item">
<a href="../../python_web/url-shorten-app/" class="md-nav__link">
The URL Shortening App
</a>
</li>
<li class="md-nav__item">
<a href="../../python_web/sre-conclusion/" class="md-nav__link">
SRE Aspects of The App and Conclusion
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--active md-nav__item--nested">
<input class="md-nav__toggle md-toggle" data-md-toggle="nav-4" type="checkbox" id="nav-4" checked>
<label class="md-nav__link" for="nav-4">
Systems Design
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" aria-label="Systems Design" data-md-level="1">
<label class="md-nav__title" for="nav-4">
<span class="md-nav__icon md-icon"></span>
Systems Design
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../intro/" class="md-nav__link">
Introduction
</a>
</li>
<li class="md-nav__item">
<a href="../scalability/" class="md-nav__link">
Scalability
</a>
</li>
<li class="md-nav__item md-nav__item--active">
<input class="md-nav__toggle md-toggle" data-md-toggle="toc" type="checkbox" id="__toc">
<label class="md-nav__link md-nav__link--active" for="__toc">
Availability
<span class="md-nav__icon md-icon"></span>
</label>
<a href="./" class="md-nav__link md-nav__link--active">
Availability
</a>
<nav class="md-nav md-nav--secondary" aria-label="Table of contents">
<label class="md-nav__title" for="__toc">
<span class="md-nav__icon md-icon"></span>
Table of contents
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="#ha-availability-common-nines" class="md-nav__link">
HA - Availability - Common “Nines”
</a>
<nav class="md-nav" aria-label="HA - Availability - Common “Nines”">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#refer" class="md-nav__link">
Refer
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#ha-availability-serial-components" class="md-nav__link">
HA - Availability Serial Components
</a>
<nav class="md-nav" aria-label="HA - Availability Serial Components">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#refer_1" class="md-nav__link">
Refer
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#ha-availability-parallel-components" class="md-nav__link">
HA - Availability Parallel Components
</a>
<nav class="md-nav" aria-label="HA - Availability Parallel Components">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#refer_2" class="md-nav__link">
Refer
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#ha-core-principles" class="md-nav__link">
HA - Core Principles
</a>
<nav class="md-nav" aria-label="HA - Core Principles">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#refer_3" class="md-nav__link">
Refer
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#ha-spof" class="md-nav__link">
HA - SPOF
</a>
</li>
<li class="md-nav__item">
<a href="#ha-reliable-crossover" class="md-nav__link">
HA - Reliable Crossover
</a>
</li>
<li class="md-nav__item">
<a href="#sre-use-cases" class="md-nav__link">
SRE Use cases
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="../fault-tolerance/" class="md-nav__link">
Fault Tolerance
</a>
</li>
<li class="md-nav__item">
<a href="../conclusion/" class="md-nav__link">
Conclusion
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle" data-md-toggle="nav-5" type="checkbox" id="nav-5" >
<label class="md-nav__link" for="nav-5">
Data
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" aria-label="Data" data-md-level="1">
<label class="md-nav__title" for="nav-5">
<span class="md-nav__icon md-icon"></span>
Data
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle" data-md-toggle="nav-5-1" type="checkbox" id="nav-5-1" >
<label class="md-nav__link" for="nav-5-1">
Big Data
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" aria-label="Big Data" data-md-level="2">
<label class="md-nav__title" for="nav-5-1">
<span class="md-nav__icon md-icon"></span>
Big Data
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../big_data/intro/" class="md-nav__link">
Introduction
</a>
</li>
<li class="md-nav__item">
<a href="../../big_data/overview/" class="md-nav__link">
Overview of Big Data
</a>
</li>
<li class="md-nav__item">
<a href="../../big_data/usage/" class="md-nav__link">
Usage of Big Data techniques
</a>
</li>
<li class="md-nav__item">
<a href="../../big_data/evolution/" class="md-nav__link">
Evolution of Hadoop
</a>
</li>
<li class="md-nav__item">
<a href="../../big_data/architecture/" class="md-nav__link">
Architecture of Hadoop
</a>
</li>
<li class="md-nav__item">
<a href="../../big_data/tasks/" class="md-nav__link">
Tasks and conclusion
</a>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle" data-md-toggle="nav-6" type="checkbox" id="nav-6" >
<label class="md-nav__link" for="nav-6">
Security
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" aria-label="Security" data-md-level="1">
<label class="md-nav__title" for="nav-6">
<span class="md-nav__icon md-icon"></span>
Security
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../security/intro/" class="md-nav__link">
Introduction
</a>
</li>
<li class="md-nav__item">
<a href="../../security/fundamentals/" class="md-nav__link">
Fundamentals of Security
</a>
</li>
<li class="md-nav__item">
<a href="../../security/network_security/" class="md-nav__link">
Network Security
</a>
</li>
<li class="md-nav__item">
<a href="../../security/threats_attacks_defences/" class="md-nav__link">
Threat, Attacks & Defences
</a>
</li>
<li class="md-nav__item">
<a href="../../security/writing_secure_code/" class="md-nav__link">
Writing Secure code
</a>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</div>
</div>
</div>
<div class="md-sidebar md-sidebar--secondary" data-md-component="toc">
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--secondary" aria-label="Table of contents">
<label class="md-nav__title" for="__toc">
<span class="md-nav__icon md-icon"></span>
Table of contents
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="#ha-availability-common-nines" class="md-nav__link">
HA - Availability - Common “Nines”
</a>
<nav class="md-nav" aria-label="HA - Availability - Common “Nines”">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#refer" class="md-nav__link">
Refer
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#ha-availability-serial-components" class="md-nav__link">
HA - Availability Serial Components
</a>
<nav class="md-nav" aria-label="HA - Availability Serial Components">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#refer_1" class="md-nav__link">
Refer
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#ha-availability-parallel-components" class="md-nav__link">
HA - Availability Parallel Components
</a>
<nav class="md-nav" aria-label="HA - Availability Parallel Components">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#refer_2" class="md-nav__link">
Refer
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#ha-core-principles" class="md-nav__link">
HA - Core Principles
</a>
<nav class="md-nav" aria-label="HA - Core Principles">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#refer_3" class="md-nav__link">
Refer
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#ha-spof" class="md-nav__link">
HA - SPOF
</a>
</li>
<li class="md-nav__item">
<a href="#ha-reliable-crossover" class="md-nav__link">
HA - Reliable Crossover
</a>
</li>
<li class="md-nav__item">
<a href="#sre-use-cases" class="md-nav__link">
SRE Use cases
</a>
</li>
</ul>
</nav>
</div>
</div>
</div>
<div class="md-content">
<article class="md-content__inner md-typeset">
<h1>Availability</h1>
<h2 id="ha-availability-common-nines">HA - Availability - Common “Nines”</h2>
<p>Availability is generally expressed as “Nines”, common ‘Nines’ are listed below.</p>
<table>
<thead>
<tr>
<th>Availability %</th>
<th align="center">Downtime per year</th>
<th align="center">Downtime per month</th>
<th align="center">Downtime per week</th>
<th align="center">Downtime per day</th>
</tr>
</thead>
<tbody>
<tr>
<td>99%(Two Nines)</td>
<td align="center">3.65 days</td>
<td align="center">7.31 hours</td>
<td align="center">1.68 hours</td>
<td align="center">14.40 minutes</td>
</tr>
<tr>
<td>99.5%(Two and a half Nines)</td>
<td align="center">1.83 days</td>
<td align="center">3.65 hours</td>
<td align="center">50.40 minutes</td>
<td align="center">7.20 minutes</td>
</tr>
<tr>
<td>99.9%(Three Nines)</td>
<td align="center">8.77 hours</td>
<td align="center">43.83 minutes</td>
<td align="center">10.08 minutes</td>
<td align="center">1.44 minutes</td>
</tr>
<tr>
<td>99.95%(Three and a half Nines)</td>
<td align="center">4.38 hours</td>
<td align="center">21.92 minutes</td>
<td align="center">5.04 minutes</td>
<td align="center">43.20 seconds</td>
</tr>
<tr>
<td>99.99%(Four Nines)</td>
<td align="center">52.60 minutes</td>
<td align="center">4.38 minutes</td>
<td align="center">1.01 minutes</td>
<td align="center">8.64 seconds</td>
</tr>
<tr>
<td>99.995%(Four and a half Nines)</td>
<td align="center">26.30 minutes</td>
<td align="center">2.19 minutes</td>
<td align="center">30.24 seconds</td>
<td align="center">4.32 seconds</td>
</tr>
<tr>
<td>99.999%(Five Nines)</td>
<td align="center">5.26 minutes</td>
<td align="center">26.30 seconds</td>
<td align="center">6.05 seconds</td>
<td align="center">864.0 ms</td>
</tr>
</tbody>
</table>
<h3 id="refer">Refer</h3>
<ul>
<li>https://en.wikipedia.org/wiki/High_availability#Percentage_calculation</li>
</ul>
<h2 id="ha-availability-serial-components">HA - Availability Serial Components</h2>
<p>A System with components is operating in the series If failure of a part leads to the combination becoming inoperable.</p>
<p>For example if LB in our architecture fails, all access to app tiers will fail. LB and app tiers are connected serially.</p>
<p>The combined availability of the system is the product of individual components availability</p>
<p><em>A = Ax x Ay x …..</em></p>
<h3 id="refer_1">Refer</h3>
<ul>
<li>http://www.eventhelix.com/RealtimeMantra/FaultHandling/system_reliability_availability.htm</li>
</ul>
<h2 id="ha-availability-parallel-components">HA - Availability Parallel Components</h2>
<p>A System with components is operating in parallel If failure of a part leads to the other part taking over the operations of the failed part.</p>
<p>If we have more than one LB and if rest of the LBs can take over the traffic during one LB failure then LBs are operating in parallel</p>
<p>The combined availability of the system is </p>
<p><em>A = 1 - ( (1-Ax) x (1-Ax) x ….. )</em></p>
<h3 id="refer_2">Refer</h3>
<ul>
<li>http://www.eventhelix.com/RealtimeMantra/FaultHandling/system_reliability_availability.htm</li>
</ul>
<h2 id="ha-core-principles">HA - Core Principles</h2>
<p><strong>Elimination of single points of failure (SPOF)</strong> This means adding redundancy to the system so that the failure of a component does not mean failure of the entire system.</p>
<p><strong>Reliable crossover</strong> In redundant systems, the crossover point itself tends to become a single point of failure. Reliable systems must provide for reliable crossover.</p>
<p><strong>Detection of failures as they occur</strong> If the two principles above are observed, then a user may never see a failure </p>
<h3 id="refer_3">Refer</h3>
<ul>
<li>https://en.wikipedia.org/wiki/High_availability#Principles</li>
</ul>
<h2 id="ha-spof">HA - SPOF</h2>
<p><strong>WHAT:</strong> Never implement and always eliminate single points of failure.</p>
<p><strong>WHEN TO USE:</strong> During architecture reviews and new designs.</p>
<p><strong>HOW TO USE:</strong> Identify single instances on architectural diagrams. Strive for active/active configurations. At the very least we should have a standby to take control when active instances fail.</p>
<p><strong>WHY:</strong> Maximize availability through multiple instances.</p>
<p><strong>KEY TAKEAWAYS:</strong> Strive for active/active rather than active/passive solutions. Use load balancers to balance traffic across instances of a service. Use control services with active/passive instances for patterns that require singletons.</p>
<h2 id="ha-reliable-crossover">HA - Reliable Crossover</h2>
<p><strong>WHAT:</strong> Ensure when system components failover they do so reliably.</p>
<p><strong>WHEN TO USE:</strong> During architecture reviews, failure modeling, and designs.</p>
<p><strong>HOW TO USE:</strong> Identify how available a system is during the crossover and ensure it is within acceptable limits. </p>
<p><strong>WHY:</strong> Maximize availability and ensure data handling semantics are preserved. </p>
<p><strong>KEY TAKEAWAYS:</strong> Strive for active/active rather than active/passive solutions, they have a lesser risk of cross over being unreliable. Use LB and right load balancing methods to ensure reliable failover. Model and build your data systems to ensure data is correctly handled when crossover happens. Generally DB systems follow active/passive semantics for writes. Masters accept writes and when master goes down, follower is promoted to master(active from being passive) to accept writes. We have to be careful here that the cutover never introduces more than one masters. This problem is called a split brain.</p>
<h2 id="sre-use-cases">SRE Use cases</h2>
<ol>
<li>SRE works on deciding an acceptable SLA and make sure system is available to achieve the SLA</li>
<li>SRE is involved in architecture design right from building the data center to make sure site is not affected by network switch, hardware, power or software failures</li>
<li>SRE also run mock drills of failures to see how the system behaves in uncharted territory and comes up with a plan to improve availability if there are misses.
https://engineering.linkedin.com/blog/2017/11/resilience-engineering-at-linkedin-with-project-waterbear</li>
</ol>
<p>Post our understanding about HA, our architecture diagram looks something like this below
<img alt="HA Block Diagram" src="../images/availability.jpg" /></p>
</article>
</div>
</div>
</main>
<footer class="md-footer">
<div class="md-footer-nav">
<nav class="md-footer-nav__inner md-grid" aria-label="Footer">
<a href="../scalability/" class="md-footer-nav__link md-footer-nav__link--prev" rel="prev">
<div class="md-footer-nav__button md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11h12z"/></svg>
</div>
<div class="md-footer-nav__title">
<div class="md-ellipsis">
<span class="md-footer-nav__direction">
Previous
</span>
Scalability
</div>
</div>
</a>
<a href="../fault-tolerance/" class="md-footer-nav__link md-footer-nav__link--next" rel="next">
<div class="md-footer-nav__title">
<div class="md-ellipsis">
<span class="md-footer-nav__direction">
Next
</span>
Fault Tolerance
</div>
</div>
<div class="md-footer-nav__button md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M4 11v2h12l-5.5 5.5 1.42 1.42L19.84 12l-7.92-7.92L10.5 5.5 16 11H4z"/></svg>
</div>
</a>
</nav>
</div>
<div class="md-footer-meta md-typeset">
<div class="md-footer-meta__inner md-grid">
<div class="md-footer-copyright">
Made with
<a href="https://squidfunk.github.io/mkdocs-material/" target="_blank" rel="noopener">
Material for MkDocs
</a>
</div>
</div>
</div>
</footer>
</div>
<script src="../../assets/javascripts/vendor.7e0ee788.min.js"></script>
<script src="../../assets/javascripts/bundle.b3a72adc.min.js"></script><script id="__lang" type="application/json">{"clipboard.copy": "Copy to clipboard", "clipboard.copied": "Copied to clipboard", "search.config.lang": "en", "search.config.pipeline": "trimmer, stopWordFilter", "search.config.separator": "[\\s\\-]+", "search.placeholder": "Search", "search.result.placeholder": "Type to start searching", "search.result.none": "No matching documents", "search.result.one": "1 matching document", "search.result.other": "# matching documents", "search.result.more.one": "1 more on this page", "search.result.more.other": "# more on this page", "search.result.term.missing": "Missing"}</script>
<script>
app = initialize({
base: "../..",
features: [],
search: Object.assign({
worker: "../../assets/javascripts/worker/search.4ac00218.min.js"
}, typeof search !== "undefined" && search)
})
</script>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink