Magento e-commerce in AKS guidance now available!

by Contributed | Nov 13, 2020 | Technology

This article is contributed. See the original author and article here.

Overview

Magento is an open-source e-commerce application based on the LAMP (Linux, Apache, MySQL, and PHP) stack. As more customers using Magento move from on-premises implementations or other cloud platforms to Azure, we wanted to provide out-of-the-box configuration recommendations to ensure easy deployment and stable, solid performance.

We spent months testing Magento performance on Azure, and we learned a lot during the process. Today we’re pleased to announce the availability of Magento e-commerce in Azure Kubernetes Service (AKS) guidance on the Azure Architecture Center. This detail is designed for developers and architects who are planning to deploy Magento on Azure.

Solution architecture

The solution architecture for Magento running in AKS is shown in the following diagram.

Key services

The key services involved in this solution architecture are described below.

• AKS deploys the Kubernetes cluster of Varnish, Magento, Redis, and Elasticsearch in different pods.


• AKS creates a virtual network to deploy the agent nodes. Create the virtual network in advance to set up subnet configuration, private link, and egress restriction.


• Varnish installs in front of the HTTP servers to act as a full-page cache.


• Azure Database for MySQL stores transaction data such as orders and catalogs. Using version 8.0 is recommended.


• Azure Files Premium or an equivalent network-attached storage (NAS) system stores media files such as product images. Magento needs a Kubernetes-compatible file system such as Azure Files Premium or Azure NetApp Files, which can mount a volume in ReadWriteMany mode.


• A content delivery network (CDN) serves static content such as CSS, JavaScript, and images. Serving content through a CDN minimizes network latency between users and the datacenter. A CDN can remove significant load from NAS by caching and serving static content.


• Redis stores session data. Hosting Redis on containers is recommended for performance reasons.


• AKS uses an Azure Active Directory (Azure AD) identity to create and manage other Azure resources like Azure load balancers, user authentication, role-based access control, and managed identity.


• Azure Container Registry stores the private Docker images that are deployed to the AKS cluster. You can use other container registries like Docker Hub. Note that the default Magento install writes some secrets to the image.


• Azure Monitor collects and stores metrics and logs, including Azure service platform metrics and application telemetry. Azure Monitor integrates with AKS to collect controller, node, and container metrics, and container and master node logs.

Key learnings

A summary of the key learnings we gained during this engagement are listed below.

• We achieved more throughput by additional components such as Varnish, Redis, etc.


• Turning off product count from layered navigation reduced the MySQL CPU consumption.


• %CPU at NFS significantly went down by caching static content in CDN.


• Turning off weblog from Varnish gave us additional performance gain.

I hope you find this guidance useful when setting up Magento in your own environment!

If you have questions or issues setting up Magento using Azure Database for MySQL, please contact the Azure Database for MySQL team at AskAzureDBforMySQL@service.microsoft.com.

Acknowledgements

My thanks go out to everyone who contributed to this project. I’d especially like to thank Darren Rich, Anil Dogra, Garima Gupta, Sumit Dua, Krishnakumar Ravi, Raj Sellappan, and Sakthi Vetrivel, who helped us conduct the performance testing and found/fixed many bottlenecks. In addition, Andrew Oakley, Lakshmikant Gundavarapu, and Parichit Sahay organized and executed countless work items to us achieve our goal.

[Announcement] Community Mentors App Update: Desktop Version Available Now!

by Contributed | Nov 13, 2020 | Technology

This article is contributed. See the original author and article here.

Have you been looking for an easier way to mentor any time and any place across your devices? Well, good news! We’ve just launched the Desktop version of the Community Mentors App. 

What is the Humans of IT Community Mentors App?

The Microsoft Humans of IT Community Mentorship App is a modern mentorship app that fosters continuous learning, authentic connection, and any-time (anywhere) access. 

Introducing More Ways to Connect! 

The Community Mentors App has added several new features helping to meet employees where they are, offering both desktop and mobile options. This month we released the highly anticipated Community Mentors App for desktop, giving mentors and mentees access to platform via web browser. The availability of both options enables everyone to engage in the most effective way.

Here are some other new features you should definitely check out! 

• Search Functionality: You are now able to search by name in the Discovery Section on Desktop 


• My Learning: Explore contact aimed at teaching us all how to have effective mentorships. 


• Reflections: Share your knowledge with the community by posting a quick reflection.

If you are on the mobile version, be sure to update to our latest version to have access to all of our new features. 

Just getting started on the app? Watch our  walkthrough demo to learn how to navigate the Community Mentors mobile app where we empower Humans of IT like you to get mentored and be mentored by other tech professionals around the world! In this video, we will walk you through how the app works, and ways you can get all set up so you can dive into the world of mentoring! 

Have ideas on new features you’d like to see, or experiences to add? Submit your ideas here, or feel free to drop us a note at msftcmp@microsoft.com.

Become a mentor/mentee on our Community Mentors app today!

1. Go to https://aka.ms/communitymentors and download our mentorship app


2. Watch our newly released Community Mentors App: Walkthrough Demo  


3. Once you’re in the app, explore new featured stories, mentorship enhancements, reactions, and notifications.


4. Check out the new desktop version at: https://aka.ms/CMPDesktop

Happy Mentoring! 

#HumansofIT

#Mentorship

#CommunityMentors

Why Restarting Azure Database for PostgreSQL took long?

by Contributed | Nov 13, 2020 | Technology

This article is contributed. See the original author and article here.

PostgreSQL is a transactional database, and it keeps a record for all DML operations and transactions like Update, Insert, and Delete in WAL (Write-Ahead Log) file. WAL logs are PostgreSQL transaction log files used to ensuring data integrity. This log is written after changes has been applied to the record and eventually once reach the checkpoint threshold the log file will be flushed from memory to storage to save it permanently.

A checkpoint is a point in the write-ahead log sequence at which all data files have been updated to reflect the information in the log. All data files will be flushed to disk. Refer to WAL Configuration for more details about what happens during a checkpoint.

Why an Azure Database for PostgreSQL restarts?

The Azure Database for PostgreSQL – Single Server service provides a guaranteed high level of availability with the financially backed service level agreement (SLA) of 99.99% uptime. Azure Database for PostgreSQL provides high availability during planned events such as user-initiated scale compute operation, and when unplanned events such as underlying hardware, software, or network failures occur. Azure Database for PostgreSQL can quickly recover from most critical circumstances, ensuring virtually no application down time when using this service. Refer to High availability in Azure Database for PostgreSQL – Single Server

Causes of server restarts:

• Planned Server Restarts

1. User initiated management operation (scale Vcores, pricing tier, etc.)


2. User Initiated restarts: Users may restart their server if they are performing a server update that requires restart such as changing a static server parameter.


3. Planned Maintenance: Azure Database for PostgreSQL performs periodic maintenance to keep your managed database secure, stable, and up to date. During maintenance, the server gets new features, updates, and patches.  Refer to: Planned maintenance notification in Azure Database for PostgreSQL. You may sign up for notification to be prepared for this scheduled maintenance using this tutorial.

• Unplanned downtime
  Unplanned downtime can occur because of unforeseen failures, including underlying hardware fault, networking issues, and software bugs. If the database server goes down unexpectedly,

What causes long recovery on Azure Database for PostgreSQL
Recent checkpoints are critical for fast server recovery. Once a restart happens, either it was a new instance (failover to healthy instance) or same instance (in-place restart) will connect to disk that has all logs, all WAL logs after the last successful checkpoint need to be applied to the data pages before the server starts to accept connections again. Those logs are called REDO logs and will be applied via the recovery operation. Applying a WAL log runs through the following steps:

1. Reading WAL file to see all the transactions inside it and all database objects and correspondently their data pages/blocks that need to be updated.


2. Fetching those pages/blocks into memory


3. Updating them using WAL file content.

Recovery time depends on how recent the last checkpoint was and the amount of inside those log files, that said, the best practice is that application developer needs to avoid log running transactions and tune checkpoint frequency to avoid long recovery.

Checkpoint Frequency:

Checkpoint frequency can be adjusted by configuring server parameters. controlled by checkpoint_completion_target which determines the total time between checkpoints. Another parameter that you may consider is bgwriter_delay which specifies the delay between activity rounds for the background writer.

Long running transactions:
Long running transactions are queries that are running for too long which impact database perfromance and can potentially cause issues during restarts. You may check all running transactions by querying pg_stat_activity, to list queries which running for more than 3 minutes, use the following query:

select current_timestamp-query_start as runtime,

datname,usename, query FROM pg_stat_activity

where state=’active’ and current_timestamp-query_start> ‘3 min’

order by 1 desc;

Please note that you can kill any long running PID using pg_terminate_backend. Let’s say you have PID “12345” and you want to kill this process, you may simply run the following query to kill it.

select pg_terminate_backend(pid)

from pg_stat_activity

where pid = ‘12345’;

If you want to kill all process on the server, run the following command:

SELECT pg_terminate_backend(pg_stat_activity.pid)

FROM pg_stat_activity

WHERE pg_stat_activity.datname = ‘TARGET_DB’ — ← change this to your DB

AND pid <> pg_backend_pid();

How to Prepare for Azure Database for PostgreSQL planned server restart

Now that we learned what causes long recovery, we will need to prepare for planned restarts either it was user initiate or system initiated.

• Ensure no long running transactions.
  As discussed earlier recovery rolls back all inflight but uncommitted transactions at the time of restart, and during this time the database is unavailable for additional requests. If there are any large transactions to rollback this activity takes time proportional to the size of the active transactions, check for long running transactions before initiating a restart and/or before maintenance window.
  
  


• Stop or reduce the application intensity workload.
  This should reduce the amount of traffic on the server, if you are planning to resize your server or know that maintenance is happening on your server, consider either stopping your application or reduce the amount of workload at that time which will significantly reduce the down time for the server.
  
  


• Force a manual Checkpoint.
  Manual checkpoint or even a recursive manual checkpoint helps tremendously in reducing recovery time. Please note that checkpoint is per database and not per server, if you have more than one database in your Azure Database for PostgreSQL server, consider running double checkpoints back-to-back before initiating the restart.

Please note if your server is under status restarting, scaling, restoring please avoid initiating another restart, this will cause longer recovery. You may check the server status on Azure Portal Overview blade for your Azure Database for PostgreSQL Server.

DevTest Labs – Shutdown Notifications in Teams Chat Messages

by Contributed | Nov 13, 2020 | Technology

This article is contributed. See the original author and article here.

Azure DevTest Labs automatic shutdown policies can save money by ensuring that VMs are shut down every night and do not sit idle indefinitely.  On those occasions when a lab user works late, the shutdown notification settings allow lab users to be warned when the machine is about to be shutdown.  In this blog post, we will cover how to use the Webhook URL setting for auto-shutdown notification settings to send a direct chat message to someone working late and warn them that their machine is about to be turned off.  We will also cover how to create the chat message so the user can delay the shutdown by an hour or two by clicking a button in the chat message.

Create Logic App to receive shutdown notifications

1. Create a Logic App.


2. Add When an HTTP request is received trigger to the Logic App.

As seen in the picture above, this action needs a JSON schema so information in the request body can be used by actions in the Logic App.  Schema for the request is below for convenience.  The Configure autoshutdown for lab and compute virtual machines in Azure DevTest Labs article contains the latest JSON schema for shutdown notifications.

  {

        “$schema”: “http://json-schema.org/draft-04/schema#“,

        “properties”: {

            “delayUrl120”: {

                “type”: “string”

            },

            “delayUrl60”: {

                “type”: “string”

            },

            “eventType”: {

                “type”: “string”

            },

            “guid”: {

                “type”: “string”

            },

            “labName”: {

                “type”: “string”

            },

            “owner”: {

                “type”: “string”

            },

            “resourceGroupName”: {

                “type”: “string”

            },

            “skipUrl”: {

                “type”: “string”

            },

            “subscriptionId”: {

                “type”: “string”

            },

            “text”: {

                “type”: “string”

            },

            “vmName”: {

                “type”: “string”

            },

            “vmUrl”: {

                “type”: “string”

            },

            “minutesUntilShutdown”: {

                “type”: “string”

            }

        },

        “required”: [

            “skipUrl”,

            “delayUrl60”,

            “delayUrl120”,

            “vmName”,

            “guid”,

            “owner”,

            “eventType”,

            “text”,

            “subscriptionId”,

            “resourceGroupName”,

            “labName”,

            “vmUrl”,

            “minutesUntilShutdown”

        ],

        “type”: “object”

    }

3. Add a Post your own adaptive card as the Flow bot to a user (preview) action to the Logic App.  This action will send a chat message from the Flow bot to a specific user.  This action also allows  an adaptive card to be sent to a user, which means we can add buttons to the message.  This action does require a connection of a Microsoft Account.

The recipient of the message should be the owner of the VM.  Get the owner’s email by searching for the ‘owner’ dynamic content from the HTTP request trigger.

The message in this action will be JSON that uses the Adaptive Card JSON schema.  For our example, we have a simple message to the user telling them that their VM will be shutdown soon and buttons to allow the user to skip the shutdown, delay the shutdown 1 hour or delay the shutdown 2 hours.

{

    “$schema”: “http://adaptivecards.io/schemas/adaptive-card.json“,

    “type”: “AdaptiveCard”,

    “version”: “1.0”,

    “speak”: “Your virtual machine is shutting down soon.  Do you want to delay the shutdown?”,

    “body”: [

        {

            “type”: “TextBlock”,

            “text”: “Virtual Machine Auto-Shutdown”,

            “size”: “large”,

            “weight”: “bolder”

        },

        {

            “type”: “TextBlock”,

            “text”: “The virtual machine @{triggerBody()[‘vmName’]} is scheduled for shutdown in @{triggerBody()?[‘minutesUntilShutdown’]} minutes.”,

            “wrap”: “true”

        }

    ],

    “actions”: [

        {

            “type”: “Action.OpenUrl”,

            “title”: “Delay 1 hour”,

            “url”: “@{triggerBody()[‘delayUrl60’]}”

        },

        {

            “type”: “Action.OpenUrl”,

            “title”: “Delay 2 hours”,

            “url”: “@{triggerBody()[‘delayUrl120’]}”

        },

        {

            “type”: “Action.OpenUrl”,

            “title”: “Skip shutdown”,

            “url”: “@{triggerBody()[‘skipUrl’]}”

        }

    ]

}

The ‘@triggerBody()’ statements tell the LogicApp to get the value from the HTTP request trigger we created in the previous step.

Lastly, set the IsAlert setting in the action to ‘Yes’.  This will cause the user to be notified in their Activity stream when the message is sent.

Action should look like the following picture.

4. Add a HTTP Response action.  Set the status code to 200 to indicate everything was successful.

Now that we have our Logic App that can handle sending a message to a user, it’s time to setup the DevTest Lab to send notifications to our Logic App.  We will need the url to call the Logic App.  To get the url, expand the When an HTTP request is received trigger step and copy the HTTP POST URL property.

Configure lab auto-shutdown settings

Auto-shutdown settings are configured at either the lab level or individual lab VM level.  Individual settings for auto-shutdown notifications are only allowed if the lab owner sets the auto-shutdown policy to allow individual users to override the lab auto-shutdown settings.  See Configure auto-shutdown for lab in Azure DevTest Labs for further details.

Let’s cover how to use the Logic App we created above by configuring auto-shutdown settings at a lab level.

1. On the home page for your lab, select Configuration and policies.


2. Select Auto-shutdown in the Schedules section of the left menu.


3. Select On to enable auto-shutdown policy.


4. For Webhook URL, paste the url for the Logic App we created earlier.


5. Select Save.

Conclusion

That’s all we need to do!  Next time a lab VM in our lab is about to be shutdown, the lab VM owner will be sent a chat message in Teams.  The message will allow the lab VM owner to quickly delay shutdown using the action buttons on the bottom of the message.