This article is contributed. See the original author and article here.
Introduction
As generative AI’s adoption rapidly expands across various industries, integrating it into products, services, and operations becomes increasingly commonplace. However, it’s crucial to address the environmental implications of such advancements, including their energy consumption, carbon footprint, water usage, and electronic waste, throughout the generative AI lifecycle. This lifecycle, often referred to as large language model operations (LLMOps), encompasses everything from model development and training to deployment and ongoing maintenance, all of which demand diligent resource optimisation.
This guide aims to extend Azure’s Well-Architected Framework (WAF) for sustainable workloads to the specific challenges and opportunities presented by generative AI. We’ll explore essential decision points, such as selecting the right models, optimising fine-tuning processes, leveraging Retrieval Augmented Generation (RAG), and mastering prompt engineering, all through a lens of environmental sustainability. By providing these targeted suggestions and best practices, we equip practitioners with the knowledge to implement generative AI not only effectively, but responsibly.
Image Description: A diagram titled “Sustainable Generative AI: Key Concepts” divided into four quadrants. Each quadrant contains bullet points summarising the key aspects of sustainable AI discussed in this article.
Select the foundation model
Choosing the right base model is crucial to optimising energy efficiency and sustainability within your AI initiatives. Consider this framework as a guide for informed decision-making:
Pre-built vs. Custom Models
When embarking on a generative AI project, one of the first decisions you’ll face is whether to use a pre-built model or train a custom model from scratch. While custom models can be tailored to your specific needs, the process of training them requires significant computational resources and energy, leading to a substantial carbon footprint. For example, training an LLM the size of GPT-3 is estimated to consume nearly 1,300 megawatt hours (MWh) of electricity. In contrast, initiating projects with pre-built models can conserve vast amounts of resources, making it an inherently more sustainable approach.
Azure AI Studio‘s comprehensive model catalogue is an invaluable resource for evaluating and selecting pre-built models based on your specific requirements, such as task relevance, domain specificity, and linguistic compatibility. The catalogue provides benchmarks covering common metrics like accuracy, coherence, and fluency, enabling informed comparisons across models. Additionally, for select models, you can test them before deployment to ensure they meet your needs. Choosing a pre-built model doesn’t limit your ability to customise it to your unique scenarios. Techniques like fine-tuning and retrieval augmented generation (RAG) allow you to adapt pre-built models to your specific domain or task without the need for resource-intensive training from scratch. This enables you to achieve highly tailored results while still benefiting from the sustainability advantages of using pre-built models, striking a balance between customisation and environmental impact.
Model Size
The correlation between a model’s parameter count and its performance (and resource demands) is significant. Before defaulting to the largest available models, consider whether more compact alternatives, such as Microsoft’s Phi-2, Mistral AI’s Mixtral 8x7B or similar sized models, could suffice for your needs. The efficiency “sweet spot”—where performance gains no longer justify the increased size and energy consumption—is critical for sustainable AI deployment. Opting for smaller, fine-tuneable models (known as small language models—or SLMs) can result in substantial energy savings without compromising effectiveness.
Model Selection
Considerations
Sustainability Impact
Pre-built Models
Leverage existing models and customise with fine-tuning, RAG and prompt engineering
Reduces training-related emissions
Custom Models
Tailor models to specific needs and customise further if needed
Higher carbon footprint due to training
Model Size
Larger models offer better output performance but require more resources
Balancing performance and efficiency is crucial
Improve the model’s performance
Improving your AI model’s performance involves strategic prompt engineering, grounding the model in relevant data, and potentially fine-tuning for specific applications. Consider these approaches:
Prompt Engineering
The art of prompt engineering lies in crafting inputs that elicit the most effective and efficient responses from your model, serving as a foundational step in customising its output to your needs. Beyond following the detailed guidelines from the likes of Microsoft and OpenAI, understanding the core principles of prompt construction—such as clarity, context, and specificity—can drastically improve model performance. Well-tuned prompts not only lead to better output quality but also contribute to sustainability by reducing the number of tokens required and the overall compute resources consumed. By getting the desired output in fewer input-output cycles, you inherently use less carbon per interaction. Orchestration frameworks like prompt flow and Semantic Kernel facilitate experimentation and refinement, enhancing prompt effectiveness with version control and reusability with templates.
Retrieval Augmented Generation (RAG)
Integrating RAG with your models taps into existing datasets, leveraging organisational knowledge without the extensive resources required for model training or extensive fine-tuning. This approach underscores the importance of how and where data is stored and accessed since its effectiveness and carbon efficiency is highly dependent on the quality and relevance of the retrieved data. End-to-end solutions like Microsoft Fabric facilitate comprehensive data management, while Azure AI Search enhances efficient information retrieval through hybrid search, combining vector and keyword search techniques. In addition, frameworks like prompt flow and Semantic Kernel enable you to successfully build RAG solutions with Azure AI Studio.
Fine-tuning
For domain-specific adjustments or to address knowledge gaps in pre-trained models, fine-tuning is a tailored approach. While involving additional computation, fine-tuning can be a more sustainable option than training a model from scratch or repeatedly passing large amounts of context via prompts and organisational data for each query. Azure OpenAI’s use of PEFT (parameter-efficient fine-tuning) techniques, like LoRA (low-rank approximation) uses far fewer computational resources over full fine-tuning. Not all models support fine-tuning so consider this in your base model selection.
Model Improvement
Considerations
Sustainability Impact
Prompt Engineering
Optimise prompts for more relevant output
Low carbon impact vs. fine-tuning, but consistently long prompts may reduce efficiency
Retrieval Augmented Generation (RAG)
Leverages existing data to ground model
Low carbon impact vs. fine-tuning, depending on relevance of retrieved data
Fine-tuning (with PEFT)
Adapt to specific domains or tasks not encapsulated in base model
Carbon impact depends on model usage and lifecycle, recommended over full fine-tuning
Deploy the model
Azure AI Studio simplifies model deployment, offering various pathways depending on your chosen model. Embracing Microsoft’s management of the underlying infrastructure often leads to greater efficiency and reduced responsibility on your part.
MaaS vs. MaaP
Model-as-a-Service (MaaS) provides a seamless API experience for deploying models like Llama 3 and Mistral Large, eliminating the need for direct compute management. With MaaS, you deploy a pay-as-you-go endpoint to your environment, while Azure handles all other operational aspects. This approach is often favoured for its energy efficiency, as Azure optimises the underlying infrastructure, potentially leading to a more sustainable use of resources. MaaS can be thought of as a SaaS-like experience applied to foundation models, providing a convenient and efficient way to leverage pre-trained models without the overhead of managing the infrastructure yourself.
On the other hand, Model-as-a-Platform (MaaP) caters to a broader range of models, including those not available through MaaS. When opting for MaaP, you create a real-time endpoint and take on the responsibility of managing the underlying infrastructure. This approach can be seen as a PaaS offering for models, combining the ease of deployment with the flexibility to customise the compute resources. However, choosing MaaP requires careful consideration of the sustainability trade-offs outlined in the WAF, as you have more control over the infrastructure setup. It’s essential to strike a balance between customisation and resource efficiency to ensure a sustainable deployment.
Model Parameters
Tailoring your model’s deployment involves adjusting various parameters—such as temperature, top p, frequency penalty, presence penalty, and max response—to align with the expected output. Understanding and adjusting these parameters can significantly enhance model efficiency. By optimising responses to reduce the need for extensive context or fine-tuning, you lower memory use and, consequently, energy consumption.
Provisioned Throughput Units (PTUs)
Provisioned Throughput Units (PTUs) are designed to improve model latency and ensure consistent performance, serving a dual purpose. Firstly, by allocating dedicated capacity, PTUs mitigate the risk of API timeouts—a common source of inefficiency that can lead to unnecessary repeat requests by the end application. This conserves computational resources. Secondly, PTUs grant Microsoft valuable insight into anticipated demand, facilitating more effective data centre capacity planning.
Semantic Caching
Implementing caching mechanisms for frequently used prompts and completions can significantly reduce the computational resources and energy consumption of your generative AI workloads. Consider using in-memory caching services like Azure Cache for Redis for high-speed access and persistent storage solutions like Azure Cosmos DB for longer-term storage. Ensure the relevance of cached results through appropriate invalidation strategies. By incorporating caching into your model deployment strategy, you can minimise the environmental impact of your deployments while improving efficiency and response times.
Model Deployment
Considerations
Sustainability Impact
MaaS
Serverless deployment, managed infrastructure
Lower carbon intensity due to optimised infrastructure
As base models evolve and user needs shift, regular assessment of model performance becomes essential. Azure AI Studio facilitates this through its suite of evaluation tools, enabling both manual and automated comparison of actual outputs against expected ones across various metrics, including groundedness, fluency, relevancy, and F1 score. Importantly, assessing performance also means scrutinising your model for risk and safety concerns, such as the presence of self-harm, hateful, and unfair content, to ensure compliance with an ethical AI framework.
Model Performance
Model deployment strategy—whether via MaaS or MaaP—affects how you should monitor resource usage within your Azure environment. Key metrics like CPU, GPU, memory utilisation, and network performance are vital indicators of your infrastructure’s health and efficiency. Tools like Azure Monitor and Azure carbon optimisation offer comprehensive insights, helping you check that your resources are allocated optimally. Consult the Azure Well-Architected Framework for detailed strategies on balancing performance enhancements with cost and energy efficiency, such as deploying to low-carbon regions, ensuring your AI implementations remain both optimal and sustainable.
A Note on Responsible AI
While sustainability is the main focus of this guide, it’s important to also consider the broader context of responsible AI. Microsoft’s Responsible AI Standard provides valuable guidance on principles like fairness, transparency, and accountability. Technical safeguards, such as Azure AI Content Safety, play a role in mitigating risks but should be part of a comprehensive approach that includes fostering a culture of responsibility, conducting ethical reviews, and combining technical, ethical, and cultural considerations. By taking a holistic approach, we can work towards the responsible development and deployment of generative AI while addressing potential challenges and promoting its ethical use.
Conclusion
As we explore the potential of generative AI, it’s clear that its use cases will continue to grow quickly. This makes it crucial to keep the environmental impact of our AI workloads in mind.
In this guide, we’ve outlined some key practices to help prioritise the environmental aspect throughout the lifecycle. With the field of generative AI changing rapidly, make sure to say up to its latest developments and keep learning.
Contributions
Special thanks to the UK GPS team who reviewed this article before it was published. In particular, Michael Gillett, George Tubb, Lu Calcagno, Sony John, and Chris Marchal.
This article is contributed. See the original author and article here.
In this era of rapid technological advancement, our industrial landscape is undergoing a significant transformation that affects many processes and people—from the way operational technology (OT) production data is leveraged to how frontline workers perform their jobs. While 2.7 billion skilled individuals keep manufacturing operations going, their attrition and retirement rates are on the rise. This heightened turnover is contributing to an ever-widening skills gap, pressuring organizations to look beyond traditional working and skilling to extend capabilities and ensure growth.
Microsoft developed Dynamics 365 Guides to address these challenges. The integration of Microsoft Copilot into Guides brings generative AI to this mixed reality solution. Copilot in Dynamics 365 Guides transforms frontline operations, putting AI in the flow of work, giving skilled and knowledge workers access to relevant information where and when they need it. This powerful combination—mixed reality together with AI—provides insight and context, allowing workers to focus on what truly matters.
Generative AI represents an enormous opportunity for manufacturers
With 63% of workers struggling to complete the repetitive tasks that take them away from more meaningful work, many are looking eagerly to technology for assistance. Generative AI addresses these realities by equipping skilled assembly, service, and knowledge workers with the information necessary to keep manufacturing moving. Integrating Copilot into Guides furthers Microsoft’s commitment to this underserved group within enterprises. Workers are using Copilot in Dynamics 365 Field Service to complete repair and service work orders faster, boosting overall productivity. Copilot is already creating efficiencies for organizations worldwide, though still in private preview, we’re excited to see how Guides unlocks frontline operations and use cases.
Copilot makes information and insight readily available. Generative AI enables Guides to put these details in context against neighboring machine components and functions, enabling technicians to repair and service faster. Copilot removes the guesswork or need to carry around those dusty old manuals. Users can ask questions using their natural language and simple gestures. Copilot summarizes relevant information to provide timely virtual guidance overlaid on top of their environment.
Manufacturers will see this innovation firsthand at Hannover Messe 2024. Partnering with Volvo Penta and BMW Group, Microsoft will illustrate generative AI’s potential on service and manufacturing frontlines. Read what we have planned at Hannover with Volvo and BMW, and what other private preview customers are doing with Copilot.
Volvo Penta is focused on transforming training in the field
Volvo Penta, a global leader in sustainable power solutions, is always looking for ways to utilize new technology to increase efficiency and accuracy and has recently been utilizing augmented reality (AR) capabilities that enhance worker training and productivity. As an early adopter of Guides and Microsoft HoloLens 2, Volvo Penta was eager to participate in the private preview for Copilot in Dynamics 365 Guides. For Volvo Penta, Copilot is another technology with the potential to unlock further value for their stakeholders.
Volvo Penta is part of a conceptual innovation exploration, to evaluate how Copilot can help optimize the training of entry-level technicians by enhancing self-guided instruction. As Volvo Penta’s Director of Diagnostics put it, “Copilot makes it feel as though a trainer is always on hand to answer questions in the context of your workflow.” Locating 10 to 15 sensors used to take new technicians an hour or more, and now it only takes five minutes. This time savings has the potential to significantly increase productivity and learning retention, helping Volvo Penta, its customers, and dealers, accomplish more. The company continues to innovate with AI and mixed reality solutions to modernize service and streamline frontline operations.
At Hannover Messe 2024, the company is showcasing how Copilot could serve their customers to improve uptime and productivity. In the demo scenario, Volvo Penta envisions its ferry captains using Copilot to address a filter issue prior to departure. Left without a service technician onboard, the captain troubleshoots replacing the filter, using Copilot and HoloLens 2 to do so with step-by-step guidance.
Volvo Penta
See how Volvo Penta streamlines frontline operations with Copilot in Dynamics 365 Guides
BMW Group is pushing the boundaries of vehicle design and development
BMW Group is improving its product lifecycle, incorporating generative AI, human-machine interactions, and software-hardware integrations for better predictability, optimization, and vehicle innovation. As a global HoloLens 2 customer, BMW Group has spent the last couple years developing its own immersive experiences and metaverse using mixed reality. Now participating in the private preview for Copilot in Dynamics 365 Guides, they are exploring how the combination of mixed reality and generative AI, together, can push the boundaries of innovation.
In private preview, BMW Group’s Digitization and virtual reality (VR) Team within research and development (R&D) is the first to evaluate Copilot’s potential on design and development. With Copilot, product designers and engineers are simulating how the use of different materials and components impact vehicle design and their environmental footprint. The insights gained through this approach will help BMW Group optimize engineering and production processes. The organization believes generative AI will also benefit its Aftersales frontline workers, providing them access to expert knowledge and guidance, whenever and where it is needed.
This joint collaboration will ultimately enable BMW Group to spark innovation and target the use cases that drive its own digital transformation forward.
Chevron is exploring the potential impact on frontline operations
AI, automation, and mixed reality solutions are poised to reshape industries everywhere. Within energy, a focus on safety and the desire to accelerate skilling has Chevron looking to advance the capabilities of its frontline workers for the future. Copilot in Dynamics 365 Guides offers Chevron the opportunity to optimize these operations, empower its workers, and infuse informed decisions throughout its value chain. AI and mixed reality, together enables Chevron to define energy in human terms.
Through the private preview for Copilot in Dynamics 365 Guides, Chevron is exploring new use cases at its El Segundo Refinery that could unlock further enhancements in worker skilling and safety.
Get started with Copilot in Dynamics 365 Guides
Interested customers can get started by deploying Dynamics 365 Guides and Dynamics 365 Remote Assist on either HoloLens 2 or mobile devices as the first step. If you want to see how AI can transform your workforce, learn how you can start implementing Microsoft Copilot today.
This article is contributed. See the original author and article here.
LeVar Burton—known for his role as Chief Engineer Geordi La Forge in Star Trek and as the host and executive producer of the beloved PBS children’s series Reading Rainbow—recently sat down for a one-on-one chatwith CVP of Microsoft Security, Vasu Jakkal, to discuss the impact of generative AI on our world.
Figure 1: LeVar Burton- pop culture icon, content creator, and literacy advocate
The conversation began with a discussion of the impact of Star Trek on both speakers’ lives. Burton spoke about how seeing actressNichelle Nichols on the bridge of the USS Enterprise meant the world to him, as it showed him what creator Gene Roddenberry said was true: “When the future came, there would be a place for me.” Jakkal shared how Star Trekwas a pivotal influence in her childhood and is in part responsible for her career in cybersecurity. “Star Trek is a perfect example of what we imagine is what we create in this realm. Human beings, we are manifesting machines,” said Burton. “And Star Trek has been responsible for helping to sow the seeds of germination for a lot of different technologies that are in use in our everyday lives today.”
Figure 2: Vasu Jakkal and LeVar Burton discussing Star Trek’s impact on technology and their hope for how generative AI will transform our world.
Generative AI (GenAI) is the transformational technology of our generation. So, we asked LeVar Burton—one of the world’s foremost storytellers and champions of learning through his work in Reading Rainbow —to help us tell the story of how GenAI will improve education and opportunities for everyone across the globe. In addition to reshaping our everyday lives, our emails, and meetings, GenAI is changing how security work gets done. These new solutions—like Microsoft Copilot for Security—help SecOps professionals make sense of large amounts of data at machine speed. They simplify the complex to help defenders find a needle in the haystack, or even a specific needle in a needle stack. Jakkal also discussed how AI can help reduce the talent shortage in the security industry and make it more diverse.
The Microsoft mission is to empower every person and organization in the world to achieve more. And the security mission is to build a safer world for all. Burton expressed his hope that generative AI will help in ways that we haven’t thought of before, referencing the cultural shift that happened in just eight nights when the groundbreaking television miniseries Roots aired. “My hope, my prayer is that generative AI can help us educate our kids in ways that we haven’t been able to and perhaps haven’t even thought of,” stated Burton. He also emphasized the importance of making GenAI safe and accessible to all. Jakkal agreed, touching on the importance of responsibility when using AI, mentioning the Microsoft responsible AI framework—a set of steps to ensure AI systems uphold six principles: fairness, reliability and safety, privacy and security, inclusiveness, transparency, and accountability.
Central to the conversation was the concept of hope, and hope for the future. Burton said the younger generation gives him hope, as they see the world and technology in a different way. Jakkal expressed her hope that we can use GenAI to change the world in a good way, by working together and being responsible. Jakkal closed the discussion by saying “I think collectively together we have to use generative AI and the technologies that we have to change this course. Storytelling, the narrative to change the narrative to one of optimism, to one of hope, to one of inclusion… for all and done by all.”
This article is contributed. See the original author and article here.
If you’re an IT administrator who manages a fleet of devices, you know how important it is to have the right tools for deployment and management. You also know how challenging it can be to find and use those tools, especially when they are scattered across different locations and versions. That’s why we are excited to announce the Surface IT Toolkit, a modern desktop application that compiles essential commercial tools and streamlines the Surface device management experience for IT admins – all in a single application.
Surface IT Toolkit is designed to provide simplified access to important tools that complement cloud and traditional management. You can follow step-by-step instructions to configure, update, or troubleshoot your devices. Not only does it provide a centralized location, the Surface IT Toolkit also ensures you are using the latest versions of our tools and installers. You don’t need to worry about outdated or incompatible versions, the app utilizes MSIX which automatically checks for updates and downloads them for you.
In this blog post, we’ll take a closer look at what’s included and what’s new with the Surface IT Toolkit.
Home
After your initial configuration, the first screen you’ll see when you open the toolkit is the home screen where you’ll find quick tasks and choose the managed devices in your fleet which helps optimize the UI of the app. You can also see the status of your tools and installers, and access the settings and feedback options.
Data Eraser
Data Eraser removes data from a Surface device using a NIST [Special Publication 800-88 Revision 1 NVM Express] format command. Additionally, it allows for the creation of certificates of sanitization for record keeping and auditing purposes. This is useful when you need to repurpose, recycle, or retire a device and ensure that no sensitive data remains on it.
What’s new with Data Eraser:
You can easily generate a certificate of sanitization after the wiping of an SSD.
The tool now provides the ability to complete a disk verification post wipe.
UEFI Configurator
UEFI Configurator lets organizations apply Surface Enterprise Management Mode (SEMM) UEFI configurations on supported devices and docks so IT staff can effectively control and deactivate components at the firmware level. This can help enhance security and compliance by preventing unauthorized changes to the device settings.
What’s new with UEFI Configurator:
In a single pass of the tool, you can create all packages needed for devices and docks (both configuration and reset).
We’re building parity between app UI and configurations historically only available to PowerShell, for example you now control USB-C ports through the UI, including USB-C Dynamic & Granular disablement.
For those utilizing PowerShell for SEMM deployment, sample PowerShell scripts are now built right into toolkit, so you quickly copy the samples into your script editor of choice and build a solution for your environment.
Recovery Tool
Recovery Tool provides the ability to perform a device reset to revert a device back to a factory state for troubleshooting scenarios. It will also help manage previously downloaded factory images to assist in re-use. This can help you resolve common issues and restore the device to its original performance.
What’s new with Recovery Tool:
Guided processes that no longer require serial numbers and simplifies the steps for building a Bare Metal Recovery (BMR) USB.
The tool provides the ability to build new and build from an existing image that you’ve already downloaded so you don’t have to fuss with version control.
We’re also providing more insight into what’s included in the image itself like what version of Windows and Microsoft 365 Apps are included.
Tool Library
Tool Library stores the latest versions of additional tools and installers that can be deployed to end users and provides IT a description of their purpose and links to supporting documentation. These include Surface Asset Tag Tool, Surface Diagnostic Toolkit for Business, Surface Brightness Control Tool, and more.
What’s new with Tool Library:
Centralized location for our other installers and tools.
Installers and tools are always up-to-date.
You can download the IT Toolkit as an MSIX package which is available from IT Pro Download Center here. You’ll also find the download link in the Surface Management and Support Suite under the Surface IT Tools section. You’ll also find the download link in the Surface Management and Support Suite under the Surface IT Tools section.
As always, be sure to check back here for more updates on managing and securing your Surface devices.
This article is contributed. See the original author and article here.
I’m excited to attend the Microsoft 365 Community Conference next week, April 30 – May 2, in Orlando, Florida with Quentin Mackey, Global Product Manager of Viva Glint, delivering a session on Holistic Listening using Viva Glint, Viva Insights, and Viva Pulse. This session will help attendees understand how to seek and act on the many signals available in the employee experience to help people feel engaged, productive, and perform at their best. We’ll be sharing best practices, showcasing new technology, and highlighting a customer case study.
There is also a track dedicated to HR professionals, communicators, and business stakeholders in employee experience empowering attendees to:
Engage employees: Inspire employees to spark participation, contribution, and action toward cultural and business objectives. Accelerate innovation and drive a high-performance organization that is inclusive of everyone from the executive suite to the frontline.
Modernize internal communications: Evolve strategies to achieve communications objectives with engaging content that reaches audiences where they work, while reducing noise & interruption. Leverage advanced analytics and AI to measure and improve effectiveness.
Join us in person with over 175 Microsoft and community experts in one place by registering here. Note: use the MSCMTY discount code to save $100 USD.
Do you want to learn more about the conference and more reasons to attend? Check out this blog to learn more about the conference.
The Microsoft 365 Community Conference returns to Orlando, FL, April 30 – May 2, 2024 – with two pre-event and one post-event workshop days. It’s a wonderful event dedicated to Copilot and AI, SharePoint, Teams, OneDrive, Viva, Power Platform, and related Microsoft 365 apps and services. Plus, a full Transformation track for communicators, HR, and business stakeholders in workplace experience.
Recent Comments