AWS IoT Greengrass deployment through Python
What Is AWS IoT Greengrass?
AWS IoT Greengrass is software that extends cloud capabilities to local devices. This enables devices to collect and analyze data closer to the source of information, react autonomously to local events, and communicate securely with each other on local networks. Local devices can also communicate securely with AWS IoT Core and export IoT data to the AWS Cloud. AWS IoT Greengrass developers can use AWS Lambda functions and prebuilt connectors to create serverless applications that are deployed to devices for local execution.
AWS IoT Greengrass makes it possible for customers to build IoT devices and application logic. Specifically, AWS IoT Greengrass provides cloud-based management of application logic that runs on devices
Major components of AWS IoT Greengrass
AWS IoT Greengrass Core
Provides local services (compute, messaging, state, security), and communicates locally with devices that run the AWS IoT Device SDK
AWS IoT Device SDK
Allows devices to interact locally with AWS IoT Greengrass Cores
AWS IoT Greengrass SDK
Allows Lambda functions to interact with local services inside an AWS IoT Greengrass Core
AWS IoT Greengrass Core devices
The AWS IoT Greengrass Core software runs on a hub, gateway, or other device to automatically sync and interact with the cloud. AWS IoT Greengrass Core is designed to run on devices with a general-purpose processor that are powerful enough to run a general-purpose operating system, such as Linux, Raspberry
AWS IoT Greengrass Groups
A Greengrass group is a collection of settings and components, such as a Greengrass core, devices, and subscriptions. Groups are used to define a scope of interaction. For example, a group might represent one floor of a building, one truck, or an entire mining site. The following diagram shows the components that can make up a Greengrass group.
In the preceding diagram:
A: Greengrass group definition
Information about group settings and components.
B: Greengrass group settings
These include:
- Greengrass group role.
- Certificate authority and local connection configuration.
- Greengrass core connectivity information.
- Default Lambda runtime environment.
· CloudWatch and local logs configuration.
C: Greengrass core
The AWS IoT thing (device) that represents the Greengrass core.
D: Lambda function definition
A list of Lambda functions that run locally on the core, with associated configuration data.
E: Subscription definition
A list of subscriptions that enable communication using MQTT messages. A subscription defines:
- A message source and message target. These can be devices, Lambda functions, connectors, AWS IoT Core, and the local shadow service.
- A topic or subject that’s used to filter messages.
F: Connector definition
A list of connectors that run locally on the core, with associated configuration data.
G: Device definition
A list of AWS IoT things (devices) that are members of the Greengrass group, with associated configuration data.
H: Resource definition
A list of local resources, machine learning resources, and secret resources on the Greengrass core, with associated configuration data.
Deploy GreenGrass core software in remote machine
The steps for deploying GG core software in remote device are
· Installs AWS IoT Greengrass dependencies.
· Downloads the root CA certificate and core device certificate and keys.
· Downloads, installs, and configures the AWS IoT Greengrass Core software on your device.
· Starts the Greengrass daemon process on the core device.
In AWS IOT Green Grass
· Creates or updates the Greengrass service role, if needed.
· Creates a Greengrass group and Greengrass core.
Python script for Deploying AWS GG Core in the remote device
def remote_connect(hostname,username,command):
try:
client = paramiko.SSHClient()
client.load_system_host_keys()
client.set_missing_host_key_policy(paramiko.WarningPolicy)
client.connect(hostname,username=username)
stdin, stdout, stderr = client.exec_command(command, get_pty=True)
output = [line.strip() for line in stdout.readlines()]
for line in output:
print(line)
print(stderr.readlines())
finally:
client.close()
def remote_ftp(hostname,username,localpath,remotepath):
try:
client = paramiko.SSHClient()
client.load_system_host_keys()
client.set_missing_host_key_policy(paramiko.WarningPolicy)
client.connect(hostname,username=username)
sftp = client.open_sftp()
#sftp.chdir(“~/aws_gg”)
sftp.put(localpath,remotepath)
if sftp.stat(remotepath):
print(“File uploaded”)
else:
print(“file didn`t upload”)
finally:
client.close()
exists = os.path.isfile(“./ggconfig.json”)
if exists:
with open(‘ggconfig.json’) as f:
custConfig = json.load(f)
region_name = custConfig[‘region’]
hostname = custConfig[‘host_name’]
print (“Creating remote directory ….”)
command = “mkdir -p ~/aws_gg”
remote_connect(hostname,username,command)
print(“creating Greengrass User and Group ….”)
command = “sudo adduser — system ggc_user”
remote_connect(hostname,username,command)
command = “sudo groupadd — system ggc_group”
remote_connect(hostname,username,command)
localpath=”./Pre_task_validation.sh”
remotepath=”/home/{0}/aws_gg/Pre_task_validation.sh”.format(username)
remote_ftp(hostname,username,localpath,remotepath)
print(“validating the user and directory …”)
command = “sudo bash ~/aws_gg/Pre_task_validation.sh”
remote_connect(hostname,username,command)
print(“installing core software in remote device …”)
localpath=”./remote_device_install_task_001.sh”
remotepath=”/home/{0}/aws_gg/remote_device_install_task_001.sh”.format(username)
remote_ftp(hostname,username,localpath,remotepath)
command = “sudo bash ~/aws_gg/remote_device_install_task_001.sh”
remote_connect(hostname,username,command)
The script uses paramiko for running remote commands and configuring prerequisites required by AWS GG core
Finally pushes the below shell script and run in the remote device either Linux or Raspberry
#!/bin/bash
cd /etc/sysctl.d
if [ ! -f 00-defaults.conf ]; then
echo “Got No file”
sudo touch 00-defaults.conf
fi
grepOutput=`grep protected_hardlinks 00-defaults.conf|wc -l`
if [ $grepOutput == 0 ]; then
echo “Got No Match”
echo ‘fs.protected_hardlinks = 1’ | sudo tee — append 00-defaults.conf
echo ‘fs.protected_symlinks = 1’ | sudo tee — append 00-defaults.conf
fi
cd ~/aws_gg
echo “Downloading core Software ………….”
curl https://raw.githubusercontent.com/tianon/cgroupfs-mount/951c38ee8d802330454bdede20d85ec1c0f8d312/cgroupfs-mount > cgroupfs-mount.sh
sudo chmod +x cgroupfs-mount.sh
sudo bash ./cgroupfs-mount.sh
echo “Installing Core software ………”
wget https://s3.amazonaws.com/ggfiles/greengrass-linux-x86-64-1.6.0.tar.gz -O gg.tar.gz
sudo tar xvzf gg.tar.gz -C /
sudo apt -y install git
git clone https://github.com/aws-samples/aws-greengrass-samples.git
sudo apt -y install python-pip
pip install awscli — upgrade — user
pip install pandas — user
pip install boto3 — user
cd /greengrass/certs/
echo “Installing Certrificate for Greengrass …”
sudo wget -O root.ca.pem https://www.amazontrust.com/repository/AmazonRootCA1.pem
cd ~/aws_gg
echo “Validation is in progress ….”
echo “Showing the Greengrass directory & files …”
ls -ltr /greengrass
echo “Showing the Greengrass certificates …”
ls -ltr /greengrass/certs/
Next am using Boto with GreenGrass SDK for creating Greengrass group and IOT policy, generating certificates
gg = boto3.client(‘greengrass’, region_name=region_name)
iot = boto3.client(‘iot’, region_name=region_name)
group_name = “your group”
group = gg.create_group(Name=group_name)
keys_cert = iot.create_keys_and_certificate(setAsActive=True)
core_thing = iot.create_thing(thingName=group_name)
iot.attach_thing_principal(
thingName=core_thing[‘thingName’],
principal=keys_cert[‘certificateArn’])
core_policy_doc = {
“Version”: “2012–10–17”,
“Statement”: [
{
“Effect”: “Allow”,
“Action”: [“iot:Publish”, “iot:Subscribe”, “iot:Connect”, “iot:Receive”, “iot:GetThingShadow”, “iot:DeleteThingShadow”, “iot:UpdateThingShadow”],
# “Resource”: [“arn:aws:iot:” + boto3.session.Session().region_name + “:*:*”]
“Resource”: [“arn:aws:iot:” + region_name + “:*:*”]
},
{
“Effect”: “Allow”,
“Action”: [“greengrass:AssumeRoleForGroup”, “greengrass:CreateCertificate”, “greengrass:GetConnectivityInfo”, “greengrass:GetDeployment”, “greengrass:GetDeploymentArtifacts”, “greengrass:UpdateConnectivityInfo”, “greengrass:UpdateCoreDeploymentStatus”],
“Resource”: [“*”]
}
]
}
policy = iot.create_policy(
policyName=group_name,
policyDocument=json.dumps(core_policy_doc))
iot.attach_principal_policy(
policyName=policy[‘policyName’],
principal=keys_cert[‘certificateArn’])
initial_version = {‘Cores’: [
{
‘Id’: core_thing[‘thingName’], # Quite intuitive, eh?
‘CertificateArn’: keys_cert[‘certificateArn’],
‘SyncShadow’: False, # Up to you, True|False
‘ThingArn’: core_thing[‘thingArn’]
}
]}
Simple Lambda creation
result = gg.create_function_definition(
Name=config[‘lambdaFunName’],
InitialVersion={
‘Functions’: [
{
‘Id’: config[‘lambdaIdName’],
‘FunctionArn’: tempARN,
‘FunctionConfiguration’: {
‘Pinned’: tempPinned,
‘MemorySize’: config[‘lambdaMemSize’],
‘Timeout’: config[‘lambdaTimeout’],
‘Environment’: {
‘AccessSysfs’: False,
‘ResourceAccessPolicies’: [
{
‘ResourceId’: config[‘LocResName’],
‘Permission’: ‘rw’
}
]
}
}
}
]
}
)
Simple subscription
result = gg.create_subscription_definition(
InitialVersion={
‘Subscriptions’: [
{
‘Id’: config[‘subscriptionIdName’],
‘Source’: tempARN,
‘Subject’: config[‘subscriptionSubject’],
‘Target’: ‘cloud’
}
]
},
Name=config[‘subscriptionName’]
)
