As documented on the main landing page for the project, GeRes2 has several components that do interact with each other through different Azure platform services. You need to create those platform services components in Azure and then configure access to those in the GeRes2 configuration files.

Pre-Requisites for Development

When developing for and with GeRes2, you’ll need the following components installed on your machine:

  • Visual Studio 2013 Professional or higher
    • Update 1 recommended but not required
  • NuGet package manager Extension installed in Visual Studio
  • Microsoft Azure SDK 2.2 or higher
    • Azure Emulator is required for local debugging
  • SQL Server Express 2008 or higher
  • A Microsoft Azure subscription
    • Also required for local debugging since GeRes2 has a dependency on Service Bus and does not work with Service Bus for Windows Server, yet.
    • We are planning to make GeRes2 compatible with the Azure Pack for Windows Server which is currently using older APIs as compared to Microsoft Azure in the public cloud depending on the release cycles and road map of the Azure pack for Windows Server.

Dependencies on Azure Platform Services & Components

In summary, the Microsoft Azure platform services leveraged and required by GeRes2 are:

Azure Service

Component

Name & Defaults

Description

Cloud Services Web/Worker Roles n/a GeRes2 is based on Microsoft Azure PaaS Cloud Services. The JobHub WebAPI and SignalR API runs in a WebRole while the JobHost which executes jobs runs in Worker Roles. The AutoScaler runs in WorkerRoles as well.
Storage Queues batch0, <batchid> Communication between JobHub and JobHost which is processing jobs. For each batch of jobs a separate queue will be created and one default queue called batch0 will always be available.
  Tables listofbatches Stores the list of batches created through the GeRes2 WebAPI.
    listofjobs Stores status and information of jobs executed in a GeRes2 deployment.
    listofjobhosts Stores status of all JobHost instances currently running or Idle and instantly available for job executions.
    roleoperationstatus Stores status of operations executed by the AutoScaler to add/remove JobHost instances for execution.
    SLABLogsTable Diagnostics logs which do store all events fired as part of logging by GeRes2. If you want all events look into the Geres.Diagnostics project in the solution folder core of the GeRes solution.
  Blobs jobprocessorfiles A blob-container storing ZIP-archives for jobs to be executed in the system. Each ZIP-archive must be stored in a separate folder. The name of the folder is the name of a “tenant” in which a job gets executed. In that folder the ZIP archive with files required for accessing the job must be stored. The files for the job must be stored in the ROOT-directory of the ZIP archive uploaded into a folder in this container.
Service Bus Topics/Subscriptions commandsforautoscaler Used for sending commands/status-updates from JobHost instances to the AutoScaler. This enables the AutoScaler to react, e.g. put instances IDLE or add/remove instances as needed.
    commandsforjobhost Used for sending commands from the AutoScaler to JobHost instances. These are especially status updates (e.g. run and actively query the job queue as fast as possible or IDLE and query the job queue in longer intervals to safe storage transaction costs).
    geres2_jobstatus Topic used to fire job-status update events to the JobHub which in turn uses SignalR-based notifications. This topic can also be used to fire events to other, integrated applications, as well. We suggest to use the Service Bus Shared Access Signature feature for such scenarios.
    geres2_canceljobs A topic which is used internally by GeRes2 to cancel jobs actively running in the system.
    signalr_topic_geres2_0 A topic which is used internally by GeRes2 to enable SignalR realtime notifications to clients and browsers subscribed to SignalR Hubs with the GeRes2 Client SDK.
Azure
Active Directory
Authentication and permissions JobHub WebAPI and consumers GeRes2 leverages Active Directory Service Credentials for authentication against the JobHub (WebAPI and SignalR).

Thus it prevents unauthorized callers from calling into the GeRes2 API when running in the public cloud.

 

For setting up an environment, whether it’s test or production, this means you need to first create storage accounts and a service bus namespace as well as configure your Azure Active Directory that can be used by GeRes2. We prepared GeRes2 to support the following setup which is also the recommendation for production environments:

  • Primary storage account: used for everything except Windows Azure Diagnostics and the table SLABLogsTable.
  • Log storage account: used for Windows Azure Diagnostics and the aforementioned SLABLogsTable as listed above.
  • One Service Bus namespace: used for all topics/subscriptions described above.
  • One Azure Active Directory: used for authorizing calls against the GeRes2 JobHub.
    • You need to register your GeRes2 JobHub as a WebAPI.
    • You need to register at least one client with a secret key used for authentication against Azure AD.

If you intend to use GeRes2 for learning or development & test environments, combining the Primary storage account and the Log storage account into a single storage account is perfectly fine. For production environments we nevertheless recommend keeping those separate so that the load of logging does only have a minimal impact on your execution performance.

Last edited Apr 30, 2014 at 12:21 PM by mszcool, version 2