diffusion_model.time_embed.0.weight diffusion_model.label_emb.0.0.

weight
(1280, 320) (1280, 2816)

diffusion_model.time_embed.0.bias diffusion_model.label_emb.0.0.bias
AccumulateGrad AccumulateGrad
(1280) (1280)

diffusion_model.time_embed.2.weight diffusion_model.label_emb.0.2.weight
AccumulateGrad ToCopyBackward0 AccumulateGrad ToCopyBackward0
(1280, 1280) (1280, 1280)

diffusion_model.time_embed.2.bias diffusion_model.label_emb.0.2.bias
ToCopyBackward0 TBackward0 AccumulateGrad ToCopyBackward0 TBackward0 AccumulateGrad
(1280) (1280)

diffusion_model.input_blocks.0.0.weight diffusion_model.input_blocks.0.0.bias
AddmmBackward0 ToCopyBackward0 AccumulateGrad AddmmBackward0 ToCopyBackward0 AccumulateGrad
(320, 16, 3, 3) (320)

AccumulateGrad AccumulateGrad SiluBackward0 TBackward0 ToCopyBackward0 SiluBackward0 TBackward0 ToCopyBackward0

ToCopyBackward0 ToCopyBackward0 AddmmBackward0 AddmmBackward0

diffusion_model.input_blocks.3.0.op.bias diffusion_model.input_blocks.3.0.op.weight
ConvolutionBackward0 AddBackward0
(320) (32, 320, 32, 32) (320, 320, 3, 3) (32, 1280)

AccumulateGrad CheckpointFunctionBackward AccumulateGrad

(32, 320, 32, 32)

CheckpointFunctionBackward ToCopyBackward0 ToCopyBackward0

ConvolutionBackward0
(32, 320, 16, 16)

diffusion_model.input_blocks.4.1.norm.weight diffusion_model.input_blocks.4.1.norm.bias
CheckpointFunctionBackward
(640) (640)

ToCopyBackward0 AccumulateGrad AccumulateGrad

NativeGroupNormBackward0

ReshapeAliasBackward0

PermuteBackward0

diffusion_model.input_blocks.4.1.proj_in.weight
ReshapeAliasBackward0
(640, 640)

diffusion_model.input_blocks.4.1.proj_in.bias
CloneBackward0 AccumulateGrad
(640)

ToCopyBackward0 AccumulateGrad ToCopyBackward0

ViewBackward0 ToCopyBackward0 TBackward0

AddmmBackward0

diffusion_model.input_blocks.4.1.proj_out.weight
ViewBackward0
(32, 77, 2048) (32, 256, 640) (640, 640)

diffusion_model.input_blocks.4.1.proj_out.bias
CheckpointFunctionBackward AccumulateGrad
(32, 256, 640) (640)

CheckpointFunctionBackward AccumulateGrad ToCopyBackward0

ViewBackward0 ToCopyBackward0 TBackward0

AddmmBackward0

ViewBackward0

ReshapeAliasBackward0

PermuteBackward0

ReshapeAliasBackward0

CloneBackward0

AddBackward0
(32, 640, 16, 16)

diffusion_model.input_blocks.5.1.norm.bias diffusion_model.input_blocks.5.1.norm.weight
CheckpointFunctionBackward
(640) (640)

AccumulateGrad AccumulateGrad ToCopyBackward0

NativeGroupNormBackward0

ReshapeAliasBackward0

PermuteBackward0

diffusion_model.input_blocks.5.1.proj_in.weight
ReshapeAliasBackward0
(640, 640)

diffusion_model.input_blocks.5.1.proj_in.bias
CloneBackward0 AccumulateGrad
(640)

ToCopyBackward0 AccumulateGrad ToCopyBackward0

ViewBackward0 ToCopyBackward0 TBackward0

AddmmBackward0

diffusion_model.input_blocks.5.1.proj_out.weight
ViewBackward0
(32, 256, 640) (640, 640)

diffusion_model.input_blocks.5.1.proj_out.bias
CheckpointFunctionBackward AccumulateGrad
(32, 256, 640) (640)

CheckpointFunctionBackward AccumulateGrad ToCopyBackward0

ViewBackward0 ToCopyBackward0 TBackward0

AddmmBackward0

ViewBackward0

ReshapeAliasBackward0

PermuteBackward0

diffusion_model.input_blocks.6.0.op.bias diffusion_model.input_blocks.6.0.op.weight
ReshapeAliasBackward0
(640) (640, 640, 3, 3)

AccumulateGrad AccumulateGrad CloneBackward0

ToCopyBackward0 ToCopyBackward0 AddBackward0

ConvolutionBackward0
(32, 640, 8, 8)

diffusion_model.input_blocks.7.1.norm.bias diffusion_model.input_blocks.7.1.norm.weight
CheckpointFunctionBackward
(1280) (1280)

AccumulateGrad AccumulateGrad ToCopyBackward0

NativeGroupNormBackward0

ReshapeAliasBackward0

PermuteBackward0

diffusion_model.input_blocks.7.1.proj_in.weight
ReshapeAliasBackward0
(1280, 1280)

diffusion_model.input_blocks.7.1.proj_in.bias
CloneBackward0 AccumulateGrad
(1280)

ToCopyBackward0 ToCopyBackward0 AccumulateGrad

ViewBackward0 TBackward0 ToCopyBackward0

AddmmBackward0

ViewBackward0
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

diffusion_model.input_blocks.7.1.proj_out.weight
CheckpointFunctionBackward
(32, 64, 1280) (1280, 1280)

diffusion_model.input_blocks.7.1.proj_out.bias
CheckpointFunctionBackward AccumulateGrad
(1280) (32, 64, 1280)

AccumulateGrad CheckpointFunctionBackward ToCopyBackward0

ToCopyBackward0 ViewBackward0 TBackward0

AddmmBackward0

ViewBackward0

ReshapeAliasBackward0

PermuteBackward0

ReshapeAliasBackward0

CloneBackward0

AddBackward0
(32, 1280, 8, 8)

diffusion_model.input_blocks.8.1.norm.bias diffusion_model.input_blocks.8.1.norm.weight
CheckpointFunctionBackward
(1280) (1280)

AccumulateGrad AccumulateGrad ToCopyBackward0

NativeGroupNormBackward0

ReshapeAliasBackward0

PermuteBackward0

diffusion_model.input_blocks.8.1.proj_in.weight
ReshapeAliasBackward0
(1280, 1280)

diffusion_model.input_blocks.8.1.proj_in.bias
CloneBackward0 AccumulateGrad
(1280)

AccumulateGrad ToCopyBackward0 ToCopyBackward0

ToCopyBackward0 ViewBackward0 TBackward0

AddmmBackward0

ViewBackward0
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

diffusion_model.input_blocks.8.1.proj_out.weight
CheckpointFunctionBackward
(32, 64, 1280) (1280, 1280)

diffusion_model.input_blocks.8.1.proj_out.bias
CheckpointFunctionBackward AccumulateGrad
(32, 64, 1280) (1280)

CheckpointFunctionBackward ToCopyBackward0 AccumulateGrad

ViewBackward0 TBackward0 ToCopyBackward0

AddmmBackward0

ViewBackward0

ReshapeAliasBackward0

PermuteBackward0

ReshapeAliasBackward0

CloneBackward0

AddBackward0
(32, 1280, 8, 8)

diffusion_model.middle_block.1.norm.weight diffusion_model.middle_block.1.norm.bias
CheckpointFunctionBackward
(1280) (1280)

AccumulateGrad AccumulateGrad ToCopyBackward0

NativeGroupNormBackward0

ReshapeAliasBackward0

PermuteBackward0

diffusion_model.middle_block.1.proj_in.weight
ReshapeAliasBackward0
(1280, 1280)

diffusion_model.middle_block.1.proj_in.bias
AccumulateGrad CloneBackward0
(1280)

ToCopyBackward0 AccumulateGrad ToCopyBackward0

TBackward0 ToCopyBackward0 ViewBackward0

AddmmBackward0

ViewBackward0
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

diffusion_model.middle_block.1.proj_out.weight
CheckpointFunctionBackward
(32, 64, 1280) (1280, 1280)

diffusion_model.middle_block.1.proj_out.bias
CheckpointFunctionBackward AccumulateGrad
(32, 64, 1280) (1280)

CheckpointFunctionBackward ToCopyBackward0 AccumulateGrad

ViewBackward0 TBackward0 ToCopyBackward0

AddmmBackward0

ViewBackward0

ReshapeAliasBackward0

PermuteBackward0

ReshapeAliasBackward0

CloneBackward0

AddBackward0
(32, 1280, 8, 8)

CheckpointFunctionBackward

CatBackward0
(32, 2560, 8, 8)

diffusion_model.output_blocks.0.1.norm.bias diffusion_model.output_blocks.0.1.norm.weight
CheckpointFunctionBackward
(1280) (1280)

AccumulateGrad AccumulateGrad ToCopyBackward0

NativeGroupNormBackward0

ReshapeAliasBackward0

PermuteBackward0

diffusion_model.output_blocks.0.1.proj_in.weight
ReshapeAliasBackward0
(1280, 1280)

diffusion_model.output_blocks.0.1.proj_in.bias
CloneBackward0 AccumulateGrad
(1280)

ToCopyBackward0 ToCopyBackward0 AccumulateGrad

ViewBackward0 TBackward0 ToCopyBackward0

AddmmBackward0

ViewBackward0
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

diffusion_model.output_blocks.0.1.proj_out.weight
CheckpointFunctionBackward
(32, 64, 1280) (1280, 1280)

diffusion_model.output_blocks.0.1.proj_out.bias
CheckpointFunctionBackward AccumulateGrad
(1280) (32, 64, 1280)

AccumulateGrad CheckpointFunctionBackward ToCopyBackward0

ToCopyBackward0 ViewBackward0 TBackward0

AddmmBackward0

ViewBackward0

ReshapeAliasBackward0

PermuteBackward0

ReshapeAliasBackward0

CloneBackward0

AddBackward0

CatBackward0
(32, 2560, 8, 8)

diffusion_model.output_blocks.1.1.norm.bias diffusion_model.output_blocks.1.1.norm.weight
CheckpointFunctionBackward
(1280) (1280)

AccumulateGrad AccumulateGrad ToCopyBackward0

NativeGroupNormBackward0

ReshapeAliasBackward0

PermuteBackward0

diffusion_model.output_blocks.1.1.proj_in.weight
ReshapeAliasBackward0
(1280, 1280)

diffusion_model.output_blocks.1.1.proj_in.bias
CloneBackward0 AccumulateGrad
(1280)

ToCopyBackward0 ToCopyBackward0 AccumulateGrad

ViewBackward0 TBackward0 ToCopyBackward0

AddmmBackward0

ViewBackward0
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

diffusion_model.output_blocks.1.1.proj_out.weight
CheckpointFunctionBackward
(32, 64, 1280) (1280, 1280)

diffusion_model.output_blocks.1.1.proj_out.bias
CheckpointFunctionBackward AccumulateGrad
(1280) (32, 64, 1280)

AccumulateGrad CheckpointFunctionBackward ToCopyBackward0

ToCopyBackward0 ViewBackward0 TBackward0

AddmmBackward0

ViewBackward0

ReshapeAliasBackward0

PermuteBackward0

ReshapeAliasBackward0

CloneBackward0

AddBackward0

CatBackward0
(32, 1920, 8, 8)

diffusion_model.output_blocks.2.1.norm.weight diffusion_model.output_blocks.2.1.norm.bias
CheckpointFunctionBackward
(1280) (1280)

AccumulateGrad AccumulateGrad ToCopyBackward0

NativeGroupNormBackward0

ReshapeAliasBackward0

PermuteBackward0

diffusion_model.output_blocks.2.1.proj_in.weight
ReshapeAliasBackward0
(1280, 1280)

diffusion_model.output_blocks.2.1.proj_in.bias
CloneBackward0 AccumulateGrad
(1280)

ToCopyBackward0 ToCopyBackward0 AccumulateGrad

ViewBackward0 TBackward0 ToCopyBackward0

AddmmBackward0

ViewBackward0
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

CheckpointFunctionBackward
(32, 64, 1280)

diffusion_model.output_blocks.2.1.proj_out.weight
CheckpointFunctionBackward
(1280, 1280) (32, 64, 1280)

diffusion_model.output_blocks.2.1.proj_out.bias
AccumulateGrad CheckpointFunctionBackward
(32, 64, 1280) (1280)

ToCopyBackward0 CheckpointFunctionBackward AccumulateGrad

TBackward0 ViewBackward0 ToCopyBackward0

AddmmBackward0

ViewBackward0

ReshapeAliasBackward0

PermuteBackward0

ReshapeAliasBackward0

diffusion_model.output_blocks.2.2.conv.bias diffusion_model.output_blocks.2.2.conv.weight
CloneBackward0
(1280) (1280, 1280, 3, 3)

AddBackward0 AccumulateGrad AccumulateGrad

UpsampleNearest2DBackward0 ToCopyBackward0 ToCopyBackward0

ConvolutionBackward0

CatBackward0
(32, 1920, 16, 16)

diffusion_model.output_blocks.3.1.norm.weight diffusion_model.output_blocks.3.1.norm.bias
CheckpointFunctionBackward
(640) (640)

AccumulateGrad AccumulateGrad ToCopyBackward0

NativeGroupNormBackward0

ReshapeAliasBackward0

PermuteBackward0

diffusion_model.output_blocks.3.1.proj_in.weight
ReshapeAliasBackward0
(640, 640)

diffusion_model.output_blocks.3.1.proj_in.bias
CloneBackward0 AccumulateGrad
(640)

AccumulateGrad ToCopyBackward0 ToCopyBackward0

ToCopyBackward0 ViewBackward0 TBackward0

AddmmBackward0

diffusion_model.output_blocks.3.1.proj_out.weight
ViewBackward0
(640, 640) (32, 256, 640)

diffusion_model.output_blocks.3.1.proj_out.bias
AccumulateGrad CheckpointFunctionBackward
(32, 256, 640) (640)

ToCopyBackward0 CheckpointFunctionBackward AccumulateGrad

TBackward0 ViewBackward0 ToCopyBackward0

AddmmBackward0

ViewBackward0

ReshapeAliasBackward0

PermuteBackward0

ReshapeAliasBackward0

CloneBackward0

AddBackward0

CatBackward0
(32, 1280, 16, 16)

diffusion_model.output_blocks.4.1.norm.bias diffusion_model.output_blocks.4.1.norm.weight
CheckpointFunctionBackward
(640) (640)

AccumulateGrad AccumulateGrad ToCopyBackward0

NativeGroupNormBackward0

ReshapeAliasBackward0

PermuteBackward0

diffusion_model.output_blocks.4.1.proj_in.weight
ReshapeAliasBackward0
(640, 640)

diffusion_model.output_blocks.4.1.proj_in.bias
AccumulateGrad CloneBackward0
(640)

ToCopyBackward0 AccumulateGrad ToCopyBackward0

TBackward0 ToCopyBackward0 ViewBackward0

AddmmBackward0

diffusion_model.output_blocks.4.1.proj_out.weight
ViewBackward0
(640, 640) (32, 256, 640)

diffusion_model.output_blocks.4.1.proj_out.bias
AccumulateGrad CheckpointFunctionBackward
(32, 256, 640) (640)

ToCopyBackward0 CheckpointFunctionBackward AccumulateGrad

TBackward0 ViewBackward0 ToCopyBackward0

AddmmBackward0

ViewBackward0

ReshapeAliasBackward0

PermuteBackward0

ReshapeAliasBackward0

CloneBackward0

AddBackward0

CatBackward0
(32, 960, 16, 16)

diffusion_model.output_blocks.5.1.norm.weight diffusion_model.output_blocks.5.1.norm.bias
CheckpointFunctionBackward
(640) (640)

AccumulateGrad AccumulateGrad ToCopyBackward0

NativeGroupNormBackward0

ReshapeAliasBackward0

PermuteBackward0

diffusion_model.output_blocks.5.1.proj_in.weight
ReshapeAliasBackward0
(640, 640)

diffusion_model.output_blocks.5.1.proj_in.bias
CloneBackward0 AccumulateGrad
(640)

ToCopyBackward0 AccumulateGrad ToCopyBackward0

ViewBackward0 ToCopyBackward0 TBackward0

AddmmBackward0

diffusion_model.output_blocks.5.1.proj_out.weight
ViewBackward0
(640, 640) (32, 256, 640)

diffusion_model.output_blocks.5.1.proj_out.bias
AccumulateGrad CheckpointFunctionBackward
(640) (32, 256, 640)

AccumulateGrad ToCopyBackward0 CheckpointFunctionBackward

ToCopyBackward0 TBackward0 ViewBackward0

AddmmBackward0

ViewBackward0

ReshapeAliasBackward0

PermuteBackward0

ReshapeAliasBackward0

diffusion_model.output_blocks.5.2.conv.bias diffusion_model.output_blocks.5.2.conv.weight
CloneBackward0
(640) (640, 640, 3, 3)

AccumulateGrad AddBackward0 AccumulateGrad

ToCopyBackward0 UpsampleNearest2DBackward0 ToCopyBackward0

ConvolutionBackward0

CatBackward0
(32, 960, 32, 32)

CheckpointFunctionBackward

CatBackward0
(32, 640, 32, 32)

CheckpointFunctionBackward

CatBackward0
(32, 640, 32, 32)

diffusion_model.out.0.weight diffusion_model.out.0.bias
CheckpointFunctionBackward
(320) (320)

ToCopyBackward0 AccumulateGrad AccumulateGrad

diffusion_model.out.2.weight diffusion_model.out.2.bias
NativeGroupNormBackward0
(12, 320, 3, 3) (12)

SiluBackward0 AccumulateGrad AccumulateGrad

ToCopyBackward0 ToCopyBackward0 ToCopyBackward0

ConvolutionBackward0

MulBackward0

AddBackward0

MeanBackward0

()