Conditional Access Overview and Templates are now Generally Available!

by Contributed | Sep 6, 2023 | Technology

This article is contributed. See the original author and article here.

————————————————————

 

Hi everyone!

 

The Conditional Access overview is a built-in dashboard that offers a comprehensive view of your Conditional Access posture. As an administrator, it provides a concise summary of your policies, identifies any gaps in your policy coverage, and provides valuable insights based on sign-in activity within your tenant. This feature enables you to swiftly pinpoint areas where you can enhance the enforcement of Zero Trust principles, ultimately bolstering your defense mechanisms.

 

Figure 1 Conditional Access overview

 

The dashboard is now the default landing page of Conditional Access. As the first entry point into Conditional Access, the overview page lets you quickly create new policies using one of the Conditional Access templates which capture commonly used policies and best practices.

 

The dashboard also offers the following insights and reporting capabilities:

 

•  The “See all unprotected sign-ins” link under the Users tile helps you rapidly identify users that are signing in without the protections of a Conditional Access policy.

 

Figure 2 Sign-ins without CA coverage during the last 7 days

 

• The ‘See all non-compliant devices’ and ‘See all unmanaged devices’ links under the Devices tile help you identify device compliance gaps.

 

Figure 3 Non-compliant devices

 

• You can discover the top 10 most accessed apps without Conditional Access coverage using the coverage tab and go one step further and identify the users without coverage for that app by clicking on the numbers in the ‘Users without coverage’ column.

 

Figure 4: Top accessed applications without CA coverage

 

You can discover security alerts generated based on sign-in activity in your tenant and take quick action on the alerts by deploying recommended zero trust conditional access policies using the Conditional Access templates. Speaking of which, let’s take a brief walkthrough of Conditional Access templates.

 

Conditional Access templates are a pre-defined set of conditions and controls that provide a convenient method to deploy new policies aligned with Microsoft recommendations. Customers are assured that their policies reflect modern best practices for securing corporate assets, promoting secure, optimal access for their hybrid workforce.

 

Conditional Access templates are organized across five scenarios:

 

• Secure foundation


• Zero Trust


• Remote work


• Protect administrators


• Emerging threats

 

Organizations can choose from 16 predefined Conditional Access templates based on their specific needs. Here is an example!

 

With the “Require phishing-resistant multifactor authentication for admins” Conditional Access template, customers can reduce the risk of compromise and phishing attacks on privileged users. This powerful template uses Conditional Access authentication strengths to help you choose the right authentication method requirements for specific scenarios, making it easier than ever for organizations to move their most critical users towards more secure, modern, and strong authentication.

 

Figure 5: Conditional Access template – Require phishing-resistant multifactor authentication for admins

 

Learn more about the Conditional Access overview dashboard: https://aka.ms/CAOverviewDashboard

 

Learn more about Conditional Access templates: https://aka.ms/ConditionalAccessTemplateDocs

 

Tell us what you think

 

Give it a try and let us know if you have questions or feedback at https://aka.ms/AzureADFeedback. We hope you will love it as much as we do!

 

Krishna Venkit

Product Manager

Microsoft Identity Division

 

Lisa Huang-North (@lisaychuang),

Senior Product Manager

Microsoft Identity Division

 

 

Learn more about Microsoft identity:   

• Get to know Microsoft Entra – a comprehensive identity and access product family   


• Return to the Microsoft Entra (Azure AD) blog home    


• Join the conversation on Twitter and LinkedIn   


• Share product suggestions on the Entra (Azure AD) forum  

Utilizando Log Analytics para monitorar logs de auditoria do Azure RedHat OpenShift

by Contributed | Sep 5, 2023 | Technology

This article is contributed. See the original author and article here.

Utilizando Log Analytics para monitorar logs de auditoria do Azure RedHat OpenShift

Introdução

Por padrão, os clusters Azure Red Hat OpenShift possuem uma forma de monitorar os logs de auditoria através do OpenShift Logging, que envolve a instalação do OpenShift Elasticsearch Operator e OpenShift Cluster Logging. Embora essa solução seja eficiente, ela não permite a integração com o Azure Monitor, a solução de monitoramento da Microsoft, nem a centralização dos logs de auditoria de diversos clusters em um único local.

Para demonstrarmos uma solução personalizada, é necessário possuir um cluster Azure Red Hat OpenShift. Caso você não possua um cluster, é possível seguir o tutorial Criando um cluster Azure Red Hat OpenShift e lembre-se de utilizar a opção do pull secret para baixar as imagens da RedHat Pull Secret

Pré-requisitos

• Ter um cluster Azure Red Hat OpenShift criado Criando um cluster Azure Red Hat OpenShift


• Ter oc client instalado instalando o oc client


• Ter o kubeclt instalado instalando o kubeclt


• Ter o Azure Cli instalado instalando o Azure Cli


• Ter o Helm Charts instalado instalando o Helm Charts

Fluent Bit

Fluent Bit é um sistema de coleta e encaminhamento de registros e logs (logs de eventos e mensagens) desenvolvido como parte do ecossistema Fluentd. É uma solução leve e eficiente projetada para coletar, filtrar e encaminhar logs em ambientes distribuídos.

Azure Red Hat OpenShift

Após a criação do cluster, vamos analisar as pastas que estão os logs de auditoria do cluster.

• Faça o login no cluster, você pode pegar o endereço do cluster no portal do Azure, na aba Overview do cluster criado e clicando no botão Connect


• Clique na URL e utilize o username kubeadmin como user e o password como senha.

Instalando o Fluent Bit no cluster

Para fazer a instalação no Azure Red Hat OpenShift, precisamos setar o security context constraints (SCC), para isso você precisa estar logado via cli e ter um usuário com permissão de cluster-admin.

Execute o comando abaixo para criar o SCC:

kubectl create -f https://raw.githubusercontent.com/fluent/fluent-bit-kubernetes-logging/master/fluent-bit-openshift-security-context-constraints.yaml

A instalação do Fluent Bit é feita via helm charts, para isso, vamos adicionar o repositório do helm charts do Fluent Bit:

helm repo add fluent https://fluent.github.io/helm-charts

Por padrão a instalação do Fluent Bit os DaemonSets são instalados somente nos workers nodes, mas para ter acesso aos logs de auditoria, precisamos fazer a instalação somente no master node, para isso, vamos criar um arquivo chamado values.yaml com o seguinte conteúdo:

# kind — DaemonSet or Deployment

kind: DaemonSet

# replicaCount — Only applicable if kind=Deployment

replicaCount: 1
image:

  repository: cr.fluentbit.io/fluent/fluent-bit

  # Overrides the image tag whose default is {{ .Chart.AppVersion }}

  tag: “latest-debug”

  pullPolicy: Always
testFramework:

  enabled: true

  image:

    repository: busybox

    pullPolicy: Always

    tag: latest
imagePullSecrets: []

nameOverride: “”

fullnameOverride: “”
serviceAccount:

  create: true

  annotations: {}

  name:
rbac:

  create: true

  nodeAccess: false
# Configure podsecuritypolicy

# Ref: https://kubernetes.io/docs/concepts/policy/pod-security-policy/

# from Kubernetes 1.25, PSP is deprecated

# See: https://kubernetes.io/blog/2022/08/23/kubernetes-v1-25-release/#pod-security-changes

# We automatically disable PSP if Kubernetes version is 1.25 or higher

podSecurityPolicy:

  create: false

  annotations: {}
openShift:

  # Sets Openshift support

  enabled: true

  # Creates SCC for Fluent-bit when Openshift support is enabled

  securityContextConstraints:

    create: true

    annotations: {}
podSecurityContext: {}

#   fsGroup: 2000
hostNetwork: false

dnsPolicy: ClusterFirst
dnsConfig: {}

#   nameservers:

#     – 1.2.3.4

#   searches:

#     – ns1.svc.cluster-domain.example

#     – my.dns.search.suffix

#   options:

#     – name: ndots

#       value: “2”

#     – name: edns0
hostAliases: []

#   – ip: “1.2.3.4”

#     hostnames:

#     – “foo.local”

#     – “bar.local”
securityContext:

  privileged: true

  runAsUser: 0

  readOnlyRootFilesystem: false

#   capabilities:

#     drop:

#     – ALL

#   readOnlyRootFilesystem: true

#   runAsNonRoot: true

#   runAsUser: 1000
service:

  type: ClusterIP

  port: 2020

  loadBalancerClass:

  loadBalancerSourceRanges: []

  labels: {}

  # nodePort: 30020

  # clusterIP: 172.16.10.1

  annotations: {}

#   prometheus.io/path: “/api/v1/metrics/prometheus”

#   prometheus.io/port: “2020”

#   prometheus.io/scrape: “true”
serviceMonitor:

  enabled: false

#   namespace: monitoring

#   interval: 10s

#   scrapeTimeout: 10s

#   jobLabel: fluentbit

#   selector:

#    prometheus: my-prometheus

#  ## metric relabel configs to apply to samples before ingestion.

#  ##

#  metricRelabelings:

#    – sourceLabels: [__meta_kubernetes_service_label_cluster]

#      targetLabel: cluster

#      regex: (.*)

#      replacement: ${1}

#      action: replace

#  ## relabel configs to apply to samples after ingestion.

#  ##

#  relabelings:

#    – sourceLabels: [__meta_kubernetes_pod_node_name]

#      separator: ;

#      regex: ^(.*)$

#      targetLabel: nodename

#      replacement: $1

#      action: replace

#  scheme: “”

#  tlsConfig: {}
  ## Beare in mind if youn want to collec metrics from a different port

  ## you will need to configure the new ports on the extraPorts property.

  additionalEndpoints: []

  # – port: metrics

  #   path: /metrics

  #   interval: 10s

  #   scrapeTimeout: 10s

  #   scheme: “”

  #   tlsConfig: {}

  #   # metric relabel configs to apply to samples before ingestion.

  #   #

  #   metricRelabelings:

  #     – sourceLabels: [__meta_kubernetes_service_label_cluster]

  #       targetLabel: cluster

  #       regex: (.*)

  #       replacement: ${1}

  #       action: replace

  #   # relabel configs to apply to samples after ingestion.

  #   #

  #   relabelings:

  #     – sourceLabels: [__meta_kubernetes_pod_node_name]

  #       separator: ;

  #       regex: ^(.*)$

  #       targetLabel: nodename

  #       replacement: $1

  #       action: replace
prometheusRule:

  enabled: false

#   namespace: “”

#   additionalLabels: {}

#   rules:

#   – alert: NoOutputBytesProcessed

#     expr: rate(fluentbit_output_proc_bytes_total[5m]) == 0

#     annotations:

#       message: |

#         Fluent Bit instance {{ $labels.instance }}’s output plugin {{ $labels.name }} has not processed any

#         bytes for at least 15 minutes.

#       summary: No Output Bytes Processed

#     for: 15m

#     labels:

#       severity: critical
dashboards:

  enabled: false

  labelKey: grafana_dashboard

  annotations: {}

  namespace: “”
lifecycle: {}

  # preStop:

  #   exec:

  #     command: [“/bin/sh”, “-c”, “sleep 20”]
livenessProbe:

  httpGet:

    path: /

    port: http
readinessProbe:

  httpGet:

    path: /api/v1/health

    port: http
resources: {}

#   limits:

#     cpu: 100m

#     memory: 128Mi

#   requests:

#     cpu: 100m

#     memory: 128Mi
## only available if kind is Deployment

ingress:

  enabled: false

  className: “”

  annotations: {}

    # kubernetes.io/ingress.class: nginx

    # kubernetes.io/tls-acme: “true”

  hosts: []

  # – host: fluent-bit.example.tld

  extraHosts: []

  # – host: fluent-bit-extra.example.tld

      ## specify extraPort number

  #   port: 5170

  tls: []

  #  – secretName: fluent-bit-example-tld

  #    hosts:

  #      – fluent-bit.example.tld
## only available if kind is Deployment

autoscaling:

  vpa:

    enabled: false
    annotations: {}
    # List of resources that the vertical pod autoscaler can control. Defaults to cpu and memory

    controlledResources: []
    # Define the max allowed resources for the pod

    maxAllowed: {}

    # cpu: 200m

    # memory: 100Mi

    # Define the min allowed resources for the pod

    minAllowed: {}

    # cpu: 200m

    # memory: 100Mi
    updatePolicy:

      # Specifies whether recommended updates are applied when a Pod is started and whether recommended updates

      # are applied during the life of a Pod. Possible values are “Off”, “Initial”, “Recreate”, and “Auto”.

      updateMode: Auto
  enabled: false

  minReplicas: 1

  maxReplicas: 3

  targetCPUUtilizationPercentage: 75

#  targetMemoryUtilizationPercentage: 75

   ## see https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/#autoscaling-on-multiple-metrics-and-custom-metrics

  customRules: []

#     – type: Pods

#       pods:

#         metric:

#           name: packets-per-second

#         target:

#           type: AverageValue

#           averageValue: 1k

    ## see https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#support-for-configurable-scaling-behavior

  behavior: {}

#      scaleDown:

#        policies:

#          – type: Pods

#            value: 4

#            periodSeconds: 60

#          – type: Percent

#            value: 10

#            periodSeconds: 60
## only available if kind is Deployment

podDisruptionBudget:

  enabled: false

  annotations: {}

  maxUnavailable: “30%”
nodeSelector:

  node-role.kubernetes.io/master: ”
tolerations:

  – key: node-role.kubernetes.io/master

    operator: Exists

    effect: NoSchedule
affinity: {}
labels: {}
annotations: {}
podAnnotations: {}
podLabels: {}
## How long (in seconds) a pods needs to be stable before progressing the deployment

##

minReadySeconds:
## How long (in seconds) a pod may take to exit (useful with lifecycle hooks to ensure lb deregistration is done)

##

terminationGracePeriodSeconds:
priorityClassName: “”
env: []

#  – name: FOO

#    value: “bar”
# The envWithTpl array below has the same usage as “env”, but is using the tpl function to support templatable string.

# This can be useful when you want to pass dynamic values to the Chart using the helm argument “–set =”

# https://helm.sh/docs/howto/charts_tips_and_tricks/#using-the-tpl-function

envWithTpl: []

#  – name: FOO_2

#    value: “{{ .Values.foo2 }}”

#

# foo2: bar2
envFrom: []
extraContainers: []

#   – name: do-something

#     image: busybox

#     command: [‘do’, ‘something’]
flush: 1
metricsPort: 2020
extraPorts: []

#   – port: 5170

#     containerPort: 5170

#     protocol: TCP

#     name: tcp

#     nodePort: 30517
extraVolumes: []
extraVolumeMounts: []
updateStrategy: {}

#   type: RollingUpdate

#   rollingUpdate:

#     maxUnavailable: 1
# Make use of a pre-defined configmap instead of the one templated here

existingConfigMap: “”
networkPolicy:

  enabled: false

#   ingress:

#     from: []
luaScripts: {}
## https://docs.fluentbit.io/manual/administration/configuring-fluent-bit/classic-mode/configuration-file

config:

  service: |

    [SERVICE]

        Daemon Off

        Flush {{ .Values.flush }}

        Log_Level {{ .Values.logLevel }}

        Parsers_File parsers.conf

        Parsers_File custom_parsers.conf

        HTTP_Server On

        HTTP_Listen 0.0.0.0

        HTTP_Port {{ .Values.metricsPort }}

        Health_Check On
  ## https://docs.fluentbit.io/manual/pipeline/inputs

  inputs: |

    [INPUT]

        Name tail

        Path /var/log/kube-apiserver/*.log

        multiline.parser docker, cri

        Tag audit.kube-apiserver.*

        DB  /tmp/kube_apiserver.db

        Mem_Buf_Limit 50MB

        Refresh_Interval  10

        Skip_Empty_Lines On

        Buffer_Chunk_Size 5M

        Buffer_Max_Size 50M

        Skip_Long_Lines Off
    [INPUT]

        Name tail

        Path /var/log/openshift-apiserver/*.log

        multiline.parser docker, cri

        Tag audit.openshift-apiserver.*

        DB  /tmp/openshift-apiserver.db

        Mem_Buf_Limit 50MB

        Refresh_Interval  10

        Skip_Empty_Lines On

        Buffer_Chunk_Size 5M

        Buffer_Max_Size 50M

        Skip_Long_Lines Off
    [INPUT]

        Name tail

        Path /var/log/oauth-apiserver/*.log

        multiline.parser docker, cri

        Tag audit.oauth-apiserver.*

        DB  /tmp/oauth-apiserver.db

        Mem_Buf_Limit 50MB

        Refresh_Interval  10

        Skip_Empty_Lines On

        Buffer_Chunk_Size 5M

        Buffer_Max_Size 50M

        Skip_Long_Lines Off
  ## https://docs.fluentbit.io/manual/pipeline/filters

  filters: |

    [FILTER]

        Name kubernetes

        Match kube.*

        Merge_Log On

        Keep_Log Off

        K8S-Logging.Parser On

        K8S-Logging.Exclude On
  ## https://docs.fluentbit.io/manual/pipeline/outputs

  outputs: |

    [OUTPUT]

        Name stdout

        Match *
  ## https://docs.fluentbit.io/manual/administration/configuring-fluent-bit/classic-mode/upstream-servers

  ## This configuration is deprecated, please use `extraFiles` instead.

  upstream: {}
  ## https://docs.fluentbit.io/manual/pipeline/parsers

  customParsers: |

    [PARSER]

        Name docker_no_time

        Format json

        Time_Keep Off

        Time_Key time

        Time_Format %Y-%m-%dT%H:%M:%S.%L
  # This allows adding more files with arbitary filenames to /fluent-bit/etc by providing key/value pairs.

  # The key becomes the filename, the value becomes the file content.

  extraFiles: {}

#     upstream.conf: |

#       [UPSTREAM]

#           upstream1

#

#       [NODE]

#           name       node-1

#           host       127.0.0.1

#           port       43000

#     example.conf: |

#       [OUTPUT]

#           Name example

#           Match foo.*

#           Host bar
# The config volume is mounted by default, either to the existingConfigMap value, or the default of “fluent-bit.fullname”

volumeMounts:

  – name: config

    mountPath: /fluent-bit/etc/fluent-bit.conf

    subPath: fluent-bit.conf

  – name: config

    mountPath: /fluent-bit/etc/custom_parsers.conf

    subPath: custom_parsers.conf
daemonSetVolumes:

  – name: varlog

    hostPath:

      path: /var/log

  – name: varlibdockercontainers

    hostPath:

      path: /var/lib/docker/containers

  – name: etcmachineid

    hostPath:

      path: /etc/machine-id

      type: File
daemonSetVolumeMounts:

  – name: varlog

    mountPath: /var/log

  – name: varlibdockercontainers

    mountPath: /var/lib/docker/containers

    readOnly: true

  – name: etcmachineid

    mountPath: /etc/machine-id

    readOnly: true
args: []
command: []
# This supports either a structured array or a templatable string

initContainers: []
# Array mode

# initContainers:

#   – name: do-something

#     image: bitnami/kubectl:1.22

#     command: [‘kubectl’, ‘version’]
# String mode

# initContainers: |-

#   – name: do-something

#     image: bitnami/kubectl:{{ .Capabilities.KubeVersion.Major }}.{{ .Capabilities.KubeVersion.Minor }}

#     command: [‘kubectl’, ‘version’]
logLevel: info

Se desejar comparar o arquivo que está sendo criado com o arquivo oficial do Fluent Bit, você pode acessar o repositório do Fluent Bit, o arquivo yaml acima também tem a configuração para as pastas abaixo de logs do Azure Red Hat OpenShift que usam a tag [INPUT].

• /var/log/kube-apiserver


• /var/log/openshift-apiserver


• /var/log/oauth-apiserver

Nessa configuração acima estão também estamos usando a imagem com a tag “latest-debug”, com essa tag é possível ver os logs do Fluent Bit no console do pod após a instalação do Fluent Bit no cluster, para isso basta executar o comando abaixo:

ls /var/log/kube-apiserver

ls /var/log/openshift-apiserver

ls /var/log/oauth-apiserver

Para realizar a instalação, esteja na mesma pasta em que o arquivo values.yaml foi criado e execute o comando abaixo.

kubectl create namespace logging

helm install fluent-bit fluent/fluent-bit –namespace logging –values values.yaml                                                                                                 

Logo após instalado, vá ao dashboard do seu cluster, selecione workloads e pods na aba lateral e selecione o project como logging, você deve ter a mesma quantidade de pods que o cluster tem de worker nodes, no meu caso são três workers nodes.

Com a configuração atual estamos somente lendo os arquivos de logs e mostrando no terminal.

Criando um Log Analytics workspace

Para enviarmos os logs para o Azure Monitor precisamos criar um Log Analytics workspace, para isso acesse siga os passos

Após a criação do Log Analytics workspace e acesse o mesmo e na menu lateral nos settings clique no Agents.

Salve o Workspace ID e o Primary Key, pois vamos usar os mesmo para a nova configuração.

Agora precisamos adicionar mais um output na configuração do ConfigMap do Fluent Bit.

• 
  

  Vá no ConfigMap(fluent-bit) e adicione o output abaixo no final do arquivo e clique no salvar.

  
  

    ## https://docs.fluentbit.io/manual/pipeline/outputs
  
      [OUTPUT]
  
          Name            azure
  
          Match           *
  
          Customer_ID     ${WorkspaceId}
  
          Shared_Key      ${SharedKey}
  
          Log_Type        AuditOpenshift

  
  


• 
  

  Execute o comando abaixo para criar uma Secret com o WorkspaceId e SharedKey (que é o seu Primary Key). Mude o xxxx para o seus valores

  
  

  kubectl create secret generic fluentbit-secret –from-literal=SharedKey=”xxxx” –from-literal=WorkspaceId=”xxxx” -n logging

  
  


• 
  

  Após criar a secret você pode verificar a mesma rodando o comando abaixo.

  
  

  kubectl get secret fluentbit-secret  -n logging

  
  


• 
  

  Agora precisamos adicionar secret no DaemonSet, para isso vá no menu lateral e selecione DaemonSets e clique no fluent-bit e  selecione Enviroments

  
  


• 
  

  

  
  


• 
  

  Clique no Add from ConfigMap or Secret

  
  


• 
  

  Adicione as environments SharedKey e WorkspaceId e no Select a resource , selecione o Secret que foi criado anteriormente fluent-bit-secret, deixe igual a imagem abaixo e clique no save.

  
  


• 
  

  Para que a nova configuração seja aplicada, é necessário excluir os Pods atuais; execute o comando abaixo.

  
  

  kubectl delete pods -l app.kubernetes.io/instance=fluent-bit  -n logging

  
  


• 
  

  Após deletar os pods, você pode verificar que os novos pods já estão sendo criados com a nova configuração, para isso execute o comando abaixo.

  
  

  kubectl get pods -l app.kubernetes.io/instance=fluent-bit  -n logging
  
  # Utilize o nome do primeiro de pod que aparecer e execute o comando abaixo para ver os logs do pod.
  
  kubectl logs fluent-bit-xxxx -n logging  | grep “customer_id=”

  
  


• 
  

  Vai mostrar os logs como abaixo, mostrando que o output para o Log Analytics workspace a foi enviado com sucesso.

  
  

    [2023/06/06 16:37:07] [ info] [output:azure:azure.1] customer_id=247446f4-e70c-4338-87d3-ba4f902a82c9, HTTP status=200
  
    [2023/06/06 16:37:07] [ info] [output:azure:azure.1] customer_id=247446f4-e70c-4338-87d3-ba4f902a82c9, HTTP status=200
  
    [2023/06/06 16:37:08] [ info] [output:azure:azure.1] customer_id=247446f4-e70c-4338-87d3-ba4f902a82c9, HTTP status=200
  
    [2023/06/06 16:37:08] [ info] [output:azure:azure.1] customer_id=247446f4-e70c-4338-87d3-ba4f902a82c9, HTTP status=200
  
    [2023/06/06 16:37:09] [ info] [output:azure:azure.1] customer_id=247446f4-e70c-4338-87d3-ba4f902a82c9, HTTP status=200

  
  

Vizualizando os logs de auditoria no Log Analytics workspace

1. 
   

   Entre no portal da azure, busque na barra de pesquisa do Log Analytics workspace e na lista selecione o Log Analytics workspace que foi criado nos passos anteriores.

   
   


2. 
   

   No menu lateral selecione logs como na imagem abaixo.

   
   

   
   
   

   
   


3. 
   

   Vai abrir uma tela de queries e feche a mesma.

   
   


4. 
   

   Em tables, abra custom logs e deve ter uma tabela com no nome AuditOpenshift_CL

   
   


5. 
   

   Vá no campo e coloque o comando abaixo e clique no Run

   
   

   AuditOpenshift_CL |
   
   take 100 

   
   


6. 
   

   Após rodar o comando, irá mostrar todos os logs de auditoria que estão sendo enviados para o Log Analytics workspace

   
   

Conclusão

Em resumo, o Fluent Bit é uma ferramenta poderosa para coletar e enviar logs para o Log Analytics Workspace da Azure. Com a configuração correta, você pode coletar logs de vários serviços e aplicativos em execução em seu cluster Kubernetes(OpenShift) e enviá-los para o Log Analytics Workspace para análise e monitoramento. Além disso, o Fluent Bit é altamente configurável e pode ser personalizado para atender às suas necessidades específicas. Esperamos que este guia tenha sido útil para você começar a usar o Fluent Bit em seu ambiente Kubernetes(OpenShift).

Referências

• Documentação do Fluent Bit


• Documentação do Azure Monitor para contêineres


• Documentação do OpenShift


• Documentação do Log Analytics Workspace

Integrating Microsoft Fabric with Azure Databricks Delta Tables

by Contributed | Sep 4, 2023 | Technology

This article is contributed. See the original author and article here.

Introduction

Microsoft Fabric is a powerful unified analytics solution that allows users to seamlessly connect to various data sources, including Azure Databricks, and create insightful reports and visualizations without the need to move the data.

In this tutorial, we’ll show you step-by-step how to connect to Azure Databricks generated Delta Tables and create a report in Microsoft Fabric.

By the end of this tutorial, you’ll have the knowledge needed to read Azure Databricks generated Delta Tables from a Microsoft Fabric using Notebook and SQL query. You will also learn how to create a Power BI report that can help drive business decisions. So, let’s get started!

Prerequisites

Before you connect, complete these steps:

• An Azure Databricks workspace


• An ADLS Gen2 account to store delta table and a parquet file


• A Fabric workspace. For more information, see Fabric trial docs.


• A lakehouse in Fabric workspace. For more information, see Create a lakehouse with OneLake.

Create Delta Table in Databricks

1. Create a delta table from Databricks Notebook.

   In this case, I am using parquet files stored in ADLS Gen2 account to create a delta table.

%SQL

--create external table from parquet files stored in ADLS Gen2
CREATE TABLE fact_internet_sales
LOCATION 'abfss://demo@[yourdatalake].dfs.core.windows.net/fact_internet_sales'
AS
SELECT * FROM PARQUET.`abfss://data@[yourdatalake].dfs.core.windows.net/fact_internet_sales`;

%Python

#Read parquet files stored in ADLS Gen2
df = spark.read.format('parquet')
    .load("abfss://data@[yourdatalake].dfs.core.windows.net/fact_internet_sales")

#Write to an external delta table
df.write 
    .format("delta") 
    .option("path", "abfss://demo@[yourdatalake].dfs.core.windows.net/fact_internet_sales") 
    .saveAsTable("fact_internet_sales")

Read the Delta Table in Fabric

Open the Fabric workspace to read, analyse and visualise the data.

Create shortcut to delta table

1. In the lakehouse, click on ellipses (…) next to the Tables and select New Shortcut.
   

    

   
   

    

   
   


2. In the New shortcut wizard, select Azure Data Lake Storage Gen2 tile. For more information, see Create an Azure Data Lake Storage Gen2 shortcut                                                                                                                
   

    

   
   


3. Enter the storage account URL in Connection settings.
   
   
   
   
   
   
   
   
   
   
   
   

   Field	Details
   URL	https://StorageAccountName.dfs.core.windows.net

   
   

                                                   

   
   


4. Enter the connection details (sign in if required) and select Next. 
   

   In this case, I am using ‘Organization Account’ Authentication kind and hence need to sign in.

   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   

   Field	Details
   Connection	Existing connections for the specified storage location will appear in the drop-down. If none exist, create a new connection.
   Connection name	The Azure Data Lake Storage Gen2 connection name.
   Authentication kind	The supported models are: Organizational account, Account Key, Shared Access Signature (SAS), and Service principal. For more information, see ADLS shortcuts.

   
   

                                                                                       

   
   


5. Enter the Shortcut Name and Sub path details and then click Create. 
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   

   Field	Details
   Shortcut Name	Name of your shortcut
   URL	The Azure Data Lake Storage Gen2 URL from the last page.
   Sub Path	The directory where the delta table resides.

   
   

   

   
   


6. The shortcut pointing to the delta table (fact_internet_sales) created in the last section will now appear as a delta table under Tables in the Explorer pane.                                                                                                        


7. 
   

   Click on the table (fact_internet_sales) and the data in the table will show up.

   
   
    
   
   
   

    

   
   

Read the data from Notebook – Lakehouse mode

The data in the table can now be queried directly from the notebook in Fabric.

1. Right-click on the table or click on ellipses (…) next to the table, click Open in notebook and then New notebook.
   
    
   
   
    
   
   
   
   
    
   
   
   
    
   
   

    

   
   


2. New notebook will appear with the query automatically generated to read the data in the table.

  Select the Run Cell button or press Ctrl+Enter to execute the query and view the data.

Read the data using SQL – SQL Endpoint mode

The data in the table can also be queried directly using T-SQL query in Fabric.

1. Browse to the SQL Endpoint created as part of Lakehouse provisioning from your workspace.


2. 
   

   After opening SQL Endpoint from the workspace, expand the database, schema and tables folder in the object Explorer to see all tables listed.

   
   


3. 
   

   Right-click on the table (fact_internet_sales) or click on ellipses (…) next to the table, click New SQL Query and then Select TOP 100 rows. 

   
   
    
   
   
   

     

   
   


4.  The script will be automatically generated and executed to show the data in the table or click on Run to execute the query.

Visualise the data in Fabric using Power BI

The data in delta table can now be accessed and analysed in Power BI. You can either create a new dataset or use the default dataset created as part of lakehouse provisioning for a new report. For more information, see Direct Lake in Power BI and Microsoft Fabric

Using new dataset

1. If you are in the Lakehouse mode, click on New Power BI dataset. 

If you are in the SQL endpoint mode, click on New Power BI dataset from Reporting tab.

2. In the New Dataset dialog, select the table to be included in the dataset and click Confirm.

3. The dataset is automatically saved in the workspace, and then opens the dataset. In the web modelling experience page, click on New Report

4. In the report authoring page, drag or select the attributes from Data pane to the left-hand side pane to be included in the visualization.

Using default dataset

1. Select your workspace and open the default dataset.

2. On the dataset page, click on Start from scratch to create a new report.

3. In the report authoring page, drag or select the attributes from Data pane to the left-hand side pane to be included in the visualization.

Summary

In conclusion, this guide provides a seamless solution for accessing Azure Databricks generated delta tables from Microsoft Fabric and visualizing the data in Power BI without the need to move the data. By following the steps outlined in this guide, you can easily connect to your delta tables and extract valuable insights from your data. With the power of Azure Databricks and Microsoft Fabric combined, you can take your data analysis to the next level and make informed decisions that drive business success.

Give it a try and let me know if this was helpful.

MVP Rudy Ooms’s Dedication Solved Real-World Problem

by Contributed | Sep 3, 2023 | Technology

This article is contributed. See the original author and article here.

Microsoft MVPs continue to help the community by sharing their profound technical knowledge and demonstrating leadership. In this article, we spotlight Rudy Ooms, an Enterprise Mobility MVP from the Netherlands, and explore his insightful troubleshooting story that made a significant impact in addressing a real-world challenge.

Rudy found a question in the Microsoft Management Customer Connection Program (MM CCP) regarding the 0x800705B4 error. The individual posting the question referenced a previous blog post by Rudy where he shared the same error, however, it was not exactly the same case. Therefore, he quickly decided to step in to help the person who posted this question.

“The moment I noticed the question popping up in the MM CCP, I became aware of the fact that the problem he was experiencing wasn’t a simple one and was giving him and his company a lot of issues and headaches. So, at that point in time, I really needed to help him out. When taking a closer look at the issue, I quickly understood that the Microsoft support desk could find it difficult to solve this case. Why? If you can’t reproduce it yourself it can become difficult to solve it and that’s where I come in”.

The issue was the device sync issue that impacted on new Autopilot enrollments due to error 0x800705b4. Rudy promptly set up his test device, started troubleshooting with his familiar tool Fiddler, went through a lot of trial and error such as using the SyncML tool, and he discovered that an illegal XML character was the culprit. By removing the assignment of the policy and the autopilot settings within the registry, he successfully mitigated the issue and made devices sync and new enrollments worked smoothly. For a comprehensive insight into his challenges and the adjustments he undertook, we highly recommend delving into the detailed narrative on his blog post. Rudy mentions that he helped another individual on the WinAdmins Discord channel facing the exact same issue.

https://call4cloud.nl/2023/06/the-0x800705b4-error-in-our-stars/

“After digging into the issue and finding the culprit within 24 hours yeah that felt pretty good,” Rudy is looking back on his contribution. Despite the lack of access to the questioner’s company’s tenants, the reason behind the early resolution of the situation was his desire to help, as well as his ability to improve his own troubleshooting skills by identifying the cause of the problem. This experience taught him a couple of things, ”It reminds me that you can learn new stuff every day… even when you thought you knew everything about SYNCML. And the MS community is strong and always willing to help – and so am I!”

The Product Group at Microsoft recognizes this wonderful troubleshooting story. Juanita Baptiste, Senior Program Manager, said of Rudy’s and the rest of the MVP community’s contributions, “The MVP community is more than just customers to bounce ideas off of. They are experts in their areas and cover more scenarios than we think. I have changed the design specs and features based on feedback from this community and it’s helped us build a better product. We can’t help everyone at the level of detail that MVPs (like Rudy) does, but the fact that they have each other for support and is an immense help to us!”

This story is the best practice of helping each other as a community. Whether you are an MVP or not, everyone has the ability to help others by sharing unique expertise and experience. Next, it is your turn. For example, the following sites (not limited to just the following, of course) can help you make a difference right away, starting today!

Microsoft Tech Community

Microsoft Q&A

GitHub

Feedback Hub

Pair programming a game with GitHub Copilot and a human

by Contributed | Sep 2, 2023 | Technology

This article is contributed. See the original author and article here.

Game rules:

Rock beats scissors (breaking it).
Scissors beat paper (cutting it).
Paper beat rock (wrapping it).
The minigame is multiplayer and the computer plays the role of your opponent and chooses a random element from the list of elements

Interaction with the player:

The console is used to interact with the player.
The player can choose one of the three options: rock, paper, or scissors.
The player can choose whether to play again.
The player should be warned if they enter an invalid option.
The player is shown their score at the end of the game.
Validation of user input:

At each round, the player must enter one of the options in the list and be informed if they won, lost, or tied with the opponent.
The minigame must handle user inputs, putting them in lowercase and informing the user if the option is invalid.
By the end of each round, the player must answer whether they want to play again or not.

# rockpaperscissor.py
   # Game rules:

    # Rock beats scissors (breaking it).
    # Scissors beat paper (cutting it).
    # Paper beat rock (wrapping it).
    # The minigame is multiplayer and the computer plays the role of your opponent and chooses a random element from the list of elements
    # Interaction with the player:
   
    # The console is used to interact with the player.
    # The player can choose one of the three options: rock, paper, or scissors.
    # The player can choose whether to play again.
    # The player should be warned if they enter an invalid option.
    # The player is shown their score at the end of the game.
    # Validation of user input:
   
    # At each round, the player must enter one of the options in the list and be informed if they won, lost, or tied with the opponent.
    # The minigame must handle user inputs, putting them in lowercase and informing the user if the option is invalid.
    # By the end of each round, the player must answer whether they want to play again or not.
   
    # game loop where the player and computer selects rock, paper, or scissors, round is repeated until player quits by typing "quit"
​

quit = False
while

not quit:

quit = False
    while not quit:
        # player input
        player = input("rock, paper, or scissors? ")
        # computer input
        import random
        computer = random.choice(["rock", "paper", "scissors"])
        # game logic
        if player == computer:
            print("tie")
        elif player == "rock":
            if computer == "paper":
                print("computer wins")
            else:
                print("player wins")
        elif player == "paper":
            if computer == "scissors":
                print("computer wins")
            else:
                print("player wins")
        elif player == "scissors":
            if computer == "rock":
                print("computer wins")
            else:
                print("player wins")
        else:
            print("invalid input")
        # ask player if they want to play again
        play_again = input("play again? (y/n) ")
        if play_again == "n":
            quit = True
        elif play_again == "y":
            quit = False
        else:
            print("invalid input")
            quit = True

One improvement you could make is to add a score counter for the player and computer, so that the player can see their score at the end of the game. Here's an updated version of the code with a score counter:

# Game rules:

# Rock beats scissors (breaking it).
# Scissors beat paper (cutting it).
# Paper beat rock (wrapping it).
# The minigame is multiplayer and the computer plays the role of your opponent and chooses a random element from the list of elements
# Interaction with the player:

# The console is used to interact with the player.
# The player can choose one of the three options: rock, paper, or scissors.
# The player can choose whether to play again.
# The player should be warned if they enter an invalid option.
# The player is shown their score at the end of the game.
# Validation of user input:

# At each round, the player must enter one of the options in the list and be informed if they won, lost, or tied with the opponent.
# The minigame must handle user inputs, putting them in lowercase and informing the user if the option is invalid.
# By the end of each round, the player must answer whether they want to play again or not.

# initialize score counters
player_score = 0
computer_score = 0

# game loop where the player and computer selects rock, paper, or scissors, round is repeated until player quits by typing "quit"
quit = False
while not quit:
    # player input
    player = input("rock, paper, or scissors? ")
    # computer input
    import random
    computer = random.choice(["rock", "paper", "scissors"])
    # game logic
    if player == computer:
        print("tie")
    elif player == "rock":
        if computer == "paper":
            print("computer wins")
            computer_score += 1
        else:
            print("player wins")
            player_score += 1
    elif player == "paper":
        if computer == "scissors":
            print("computer wins")
            computer_score += 1
        else:
            print("player wins")
            player_score += 1
    elif player == "scissors":
        if computer == "rock":
            print("computer wins")
            computer_score += 1
        else:
            print("player wins")
            player_score += 1
    else:
        print("invalid input")
    # ask player if they want to play again
    play_again = input("play again? (y/n) ")
    if play_again == "n":
        quit = True
    elif play_again == "y":
        quit = False
    else:
        print("invalid input")
        quit = True

# print final scores
print("player score:", player_score)
print("computer score:", computer_score)

improve code by adding a rule matrix to compare player and computer choices

    # Game rules:

# Rock beats scissors (breaking it).
# Scissors beat paper (cutting it).
# Paper beat rock (wrapping it).
# The minigame is multiplayer and the computer plays the role of your opponent and chooses a random element from the list of elements
# Interaction with the player:

# The console is used to interact with the player.
# The player can choose one of the three options: rock, paper, or scissors.
# The player can choose whether to play again.
# The player should be warned if they enter an invalid option.
# The player is shown their score at the end of the game.
# Validation of user input:

# At each round, the player must enter one of the options in the list and be informed if they won, lost, or tied with the opponent.
# The minigame must handle user inputs, putting them in lowercase and informing the user if the option is invalid.
# By the end of each round, the player must answer whether they want to play again or not.

# initialize score counters
player_score = 0
computer_score = 0

# rule matrix
rules = {
    "rock": {"rock": "tie", "paper": "lose", "scissors": "win"},
    "paper": {"rock": "win", "paper": "tie", "scissors": "lose"},
    "scissors": {"rock": "lose", "paper": "win", "scissors": "tie"}
}

# game loop where the player and computer selects rock, paper, or scissors, round is repeated until player quits by typing "quit"
quit = False
while not quit:
    # player input
    player = input("rock, paper, or scissors? ")
    # computer input
    import random
    computer = random.choice(["rock", "paper", "scissors"])
    # game logic
    if player not in rules:
        print("invalid input")
    else:
        result = rules[player][computer]
        if result == "win":
            print("player wins")
            player_score += 1
        elif result == "lose":
            print("computer wins")
            computer_score += 1
        else:
            print("tie")
    # ask player if they want to play again
    play_again = input("play again? (y/n) ")
    if play_again == "n":
        quit = True
    elif play_again == "y":
        quit = False
    else:
        print("invalid input")
        quit = True

# print final scores
print("player score:", player_score)
print("computer score:", computer_score)