Updates

• Keep podGracefulDeletionTimeoutSec as Nullable (#74)

Full Commit History Since Previous Release

Updates

• Switch to go mod (#69)
• Upgrade CRD to apiextensions.k8s.io/v1 to support k8s >= 1.22 (#72)

Full Commit History Since Previous Release

New Feature

• Expose Task History (#62)

Full Commit History Since Previous Release

New Feature

• Expose and increase default sync concurrency (#60)
• Treat invalid Pod caused by network error as PodCreationUnknownError (#61)

Full Commit History Since Previous Release

New Feature

• Support Framework ScaleUp/ScaleDown with Strong Safety Guarantee
• Support Create ExecutionType: Just create without start

Full Commit History Since Previous Release

New Feature

• Enrich PodSpecError to early fail Pod (#52)

Bug Fix

• Fix invalid json in log caused by fmt (MISSING) (#49)
• Aware UID change during Update event and Sync (#51)

Full Commit History Since Previous Release

New Feature

• Support large scale Framework by LargeFrameworkCompression (#44)
• Add PodNodeName to help track failures on node before PodIP is available (#45)

Full Commit History Since Previous Release

New Feature

• Add example to leverage HivedScheduler to achieve GPU Topology-Aware, Multi-Tenant, Priority and Gang Scheduling (#34)
• Support to expose Framework and Pod history snapshots to external systems (#31)
• Support to classify and summarize Pod failures (#41)
• Support to tune Framework Consistency vs Availability (#43)
  This helps to avoid the Pod is stuck in deleting forever, such as if its Node is down forever.
• Support Stop Framework (#24)
  This helps to stop the Framework without deleting it.
• Still sync Task after FrameworkAttempt Completing (#27)
  This helps to make sure all Tasks in the Framework are updated to the right completed status when the whole FrameworkAttempt is completed.
• Support FrameworkCompletedRetainSec (#37)
  This helps to automatically delete the Framework after it is completed for a long time, to free ETCD space.
• Add FrameworkAttemptPreparing State (#12)
  This helps to distinguish if there is at least one Task of current attempt has ever entered TaskAttemptRunning state. If not, it is FrameworkAttemptPreparing instead of FrameworkAttemptRunning anymore.
• Redefine FrameworkAttemptRunning and Record attempt running start time (#35)
  This helps to measure Framework and Task pure running duration.
• Support Pod Template Placeholders (#21)

Bug Fix

• Fix TaskCompleted may transition to TaskAttemptCompleted (#10)
• Fix fExpectedStatusInfos map race condition (#18)

Misc

• Upgrade to kubernetes-1.14.2 (#16)
• Remove Internal and External CompletionTypeAttribute (#22 #41)
  This is because FrameworkController does not need to aware it, so leave the freedom to controller wrapper
• Upgrade to golang 1.12.6 (#29)
• Switch to klog (#30)

Full Commit History Since Previous Release

Breaking Change

1. Refine AnnotationKey, LabelKey and EnvName

Refine Doc and Example

1. Provide Doc and Example for Authentication and Authorization Enabled Cluster
2. Update NNI Controller Wrapper to Released state

Add Distributed TensorFlow Training Example

Feature

1. Support both GPU and CPU Distributed Training
2. Automatically clean up PS when the whole FrameworkAttempt is completed
3. No need to adjust existing TensorFlow image
4. No need to setup Kubernetes DNS and Kubernetes Service
5. Common Feature

Prerequisite

1. Need to setup Kubernetes GPU, if you need GPU Training
2. Need to setup Kubernetes Cluster-Level Logging, if you need to persist and expose the log for deleted Pod

Quick Start

1. Common Quick Start
2. Tensorflow Example

Support FrameworkBarrier for GangExecution

Feature

It is usually used as the InitContainer to provide a simple way to

1. Do Gang Execution without resource deadlock
2. Start the AppContainers in the Pod only after its PodUID is persisted by FrameworkController
3. Inject peer-to-peer service discovery information into the AppContainers

Quick Start

1. FrameworkBarrier User Manual