Design a Zero Trust pipeline using Confidential Computing

 

Check out reference diagram to design a zero trust pipeline

https://blogs.oracle.com/cloud-infrastructure/designing-zero-trust-and-resilient-pipelines-for-healthcare-workloads-on-oci-using-confidential-computing

Confidential Computing: How to get started

If you want to lean fundamentals of confidential computing and want to get started , check out  this blog 

https://blogs.oracle.com/cloud-infrastructure/protecting-sensitive-healthcare-workloads-with-confidential-computing-on-oci

LLM Evaluation Fundamental techniques

In lot of my conversation  with fellow technologist one question which is mostly in everyones mind is "How to make sure my application is predictable" .

I thought of sharing my learning :

https://www.linkedin.com/pulse/mental-model-llm-application-evaluation-moving-beyond-raghu-iyer-webrc/?trackingId=pIuqdLvHbk0P551bDTApUg%3D%3D

Handle the error “OwnershipControlsNotFoundError” for buckets created prior to April 2023 when the bucket ownership was defaulted to “Object Writer”

Here you go published a  python code sample written for migrating S3 bucket ownership controls between AWS accounts. The code is written to help developers handle the error “OwnershipControlsNotFoundError” when get-bucket-ownership-controls API is used to read the S3 bucket ownership from buckets created prior to April 2023 when the bucket ownership was defaulted to “Object Writer”

Any suggestion on improvement , leave comment.

https://github.com/aws-samples/sample-migratings3bucketownership

Building resilient applications: design patterns for handling database outages

Check the blog authored by me on design patterns . Design pattern for any application is not a after thought, it is critical to think through before design is finalized.

https://aws.amazon.com/blogs/database/building-resilient-applications-design-patterns-for-handling-database-outages/

Gray Failures: What is it and how to detect one?

If you are reading this article , i guess you are curious to know about gray failures and different methods to detect gray failures. 

Hopefully this article will give you insight on question in your mind. 

https://aws.amazon.com/blogs/mt/detecting-gray-failures-with-outlier-detection-in-amazon-cloudwatch-contributor-insights/

How to select the best machine learning service in AWS

 

If you looking for directional information on how to select an Machine learning algorithm, check the below blog.   The blog i wrote though is specific for manufacturing but it can provide direction on different use cases

https://aws.amazon.com/blogs/industries/selecting-the-best-automatic-machine-learning-to-meet-your-manufacturing-needs/

Testing AWS Cloudformation Template

 Very interesting tool to test the cloud formation templates. Check it out 

TaskCat

https://github.com/aws-ia/taskcat

Architecting Resilient application on AWS

 Great whitepaper on improving resiliency of distributed systems:

https://docs.aws.amazon.com/whitepapers/latest/availability-and-beyond-improving-resilience/availability-and-beyond-improving-resilience.html

AWS CDK Samples

 Want to learn AWS CDK with samples :

Check these out:

https://github.com/aws-samples/aws-cdk-examples#Examples

https://cdkpatterns.com/patterns/all/

Tranformation in Manufacturing

 Do you want to know how CPG industry  is undergoing transformation using IoT technology, read the blogs on how AWS partners are using AWS technologies to build solutions for CPG industry

https://aws.amazon.com/blogs/industries/how-to-build-a-smart-production-solution-with-dxcs-spark-platform/

Developing Custom web portal for the AWS SiteWise Service

 Are you looking to develop a custom application to show the data from AWS SiteWise?

Check out the AWS IoT Application Kit

AWSIoTPythonSDK.exception.AWSIoTExceptions.connectTimeoutException

Are you getting the below error when sening data to IoT Core using the "AWSIoTPythonSDK." ?

Here is my error :

  File "/usr/local/lib/python3.6/site-packages/AWSIoTPythonSDK/MQTTLib.py", line 520, in connect
    return self._mqtt_core.connect(keepAliveIntervalSecond)
  File "/usr/local/lib/python3.6/site-packages/AWSIoTPythonSDK/core/protocol/mqtt_core.py", line 199, in connect
    raise connectTimeoutException()
AWSIoTPythonSDK.exception.AWSIoTExceptions.connectTimeoutException

The below can be reason for  this issue

1- You have not defined policy to allow the sending message to the IoT Core topic. If that is the case create a policy for the "Thing"

You need to define

  Publish/Subscribe policy 

Connect and Publish Policy

 

If the policies are defined than check the active version of policy, In my case old policy was active

 

2- Make the latest one active  

 Check this  under your "Thing" policy settings:

 

 

 Hopefully one of it works. Leave comment if it does help

Thanks

 

Temporary IAM credential for non AWS deployed application/ service - IAM Roles Anywhere

 Tired of managing long term IAM credential for connecting to AWS service from external application? Challenged with managing client ID and secret? Your security team not comfortable  you managing an credential for critical features?

 

Wait no more. 

Check out the new feature IAM roles anywhere which i have been missing all these years as managing long term IAM credential has always been an hassle and not secure.

https://aws.amazon.com/blogs/security/extend-aws-iam-roles-to-workloads-outside-of-aws-with-iam-roles-anywhere/

AWS Amplify : Failed to pull sandbox app, reason: unable to get local issuer certificate

Here is the scenario: 

You are trying to pull Sandbox app from Amplify and complete data model using the command:

amplify pull --sandboxId "AppGUID."

and you get the error message :

"Failed to pull sandbox app: request to https://production.us-east-1.appstate.amplifyapp.com/AppState/"AppGUID" failed, reason: unable to get local issuer certificate"

The solution which generally works:

Log off company VPN if you are connecting to your organization's VPN. You will find this solution when you google the error. Well, problem solved, RIGHT?  

It worked for me, and I was able to pull the application to local with the complete data model.

Curious and Open Question  :

But the question is why it even happened ????? Please read if you are curious and not just looking for an easy solution without knowing the actual cause. 

Rationale:

I performed a few tests using OpenSSL to understand the complete communication between client and server. One of the simpler commands you can use is the -connect command:

openssl s_client -connect production.us-east-1.appstate.amplifyapp.com:443 

After reading the article below, I learned that the proxy servers don't perform automatic checks for certificate chaining like browsers do by using the AIA (Authority Information Access) field in the certificate. 

refer to this article for intermediate certificate :https://blogs.cisco.com/security/where-is-my-intermediate-tls-certificate

The leading cause seems to missing intermediate certificates in the proxy server, hence the error "unable to get local issuer certificate." In the case of proxy servers, one solution is to import the missing certificates to the device certificate store to get it working.

Hope this explains the root cause of the issue, which is generic and not specific to only this scenario.

Mechanical Sympathy - Closing post 2021

 

One of my techno-geek followers sent a note a  few weeks back asking for updates on my last post for this year and asked about my favorite post. Unfortunately, I have not been actively blogging in 2021, said that I  have been working offline with the engineering community actively on exploring new technologies. One of my favorites has been learning the machine learning area.  I would like to thank all my fellow engineers for their collaboration and engagement over the years. 

As we move on to 2022, I thought of looking back for my favorite post over the years and this post beats all others and is still my favorite. 

Quote posted around four years back from Jackie Stewart "https://www.metahat.net/2018/03/mechanical-sympathy-learning-from-racing.html"

“You don’t have to be an engineer to be a racing driver, but you do have to have Mechanical Sympathy.” 

– Jackie Stewart, racing driver

I wish all of you, Happy and Healthy New Year!!!

Why do we use np.random.seed()

This is a simple function but to my surprise lot of ML engineers had asked me about the actual usage of this function and why do I use it when i am working on an algorithm which will be shared with other engineers. 

The simple answer is :

the function generates a random deterministic number. It generated the same random number each time for a specified seed  

"It's a random function with deterministic output for a specified seed. This helps to share the code and get the same output each time. "

Try this :

np.random.seed(1)

np.random.randint(low = 1, high = 10, size = 20)

The output will be, each time till seed is specified as "1" 

array([6, 9, 6, 1, 1, 2, 8, 7, 3, 5, 6, 3, 5, 3, 5, 8, 8, 2, 8, 1])

Installing Octave in Ubuntu running on Docker

I want to run Octave on Ubuntu running on docker.  Simple stuff but still manage to get errors and had to spend  few minutes on fine-tuning 

Here it goes

1- Start docket and get Ubuntu image from repo

    - Couple of commands

            - docker pull ubuntu

            - docker run -it ubuntu 

2- Install Octave 

    - ran command "apt-get install octave" and here it goes error "Unable to locate package Octave"

  

3- to come over this issue run below commands

    - apt-get update

    - apt-get install octave 

Done......

Integration Design Best Practice

Over the past few years of working on multiple critical systems solution design initiatives, there is one important thing I have learnt. 

" There is a reason we call it solution design and not system design". With the influx of cloud offerings and services, the organization no more rely on one system to automate the business process. Rather business process now relies on multiple systems performing a very specific step of the business process. All these systems are integrated to provide business process automation. This makes integration, not one of the factors but a very critical backbone of the business process." 

Nothing that I said is new and no one will disagree but unfortunately, integrations are still given less priority and design thought. 

I keep getting one question all the time " What are integration design best practices and how can I design a good reliable integration".  I have a long list of to-dos for designing good integration but sometimes being too prescriptive is not useful as it hinders a thinking ability of an architect.  

So if you are still reading and curious to know how to design a good integration than there is only one fact you need to be aware of, this will influence your design decision and push you to design better integrations. 

AND THAT IS 

Design with the above fact in mind and try to make your integration overcome failure and you will have a good design in place. Please don't spend time on developing integrations that will not fail. Such a design is purely a myth. 

Now let's go one more level down, I have put down design pillars which will help to design the integration.   These design pillars apply to any software development initiatives 

 

if this is helpful, do leave our comment. I would love to hear your design learnings. 

No supported authentication methods available (server sent: publickey)

---------------------------
PuTTY Fatal Error
---------------------------
PuTTY Fatal Error Disconnected: No supported authentication methods available (server sent: publickey)
---------------------------


Scenario : 
Error while connecting to AWS AMI Instance using PuTTy.


Solution:
in my case "admin" user worked.
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/TroubleshootingInstancesConnecting.html#TroubleshootingInstancesConnectingPuTTY