FIT Profiler Report

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                                                                     	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                                                                      	|  -              	|  881425         	|  755.61         	|  100 %          	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                                                                         	|  35.618         	|  20             	|  712.35         	|  94.275         	|
|  run_training_batch                                                                                                                                                                                         	|  0.031577       	|  20000          	|  631.54         	|  83.58          	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                                                                 	|  0.031352       	|  20000          	|  627.04         	|  82.984         	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                                                               	|  0.010453       	|  20000          	|  209.06         	|  27.668         	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                                                                    	|  0.0017593      	|  20000          	|  35.186         	|  4.6566         	|
|  [LightningModule]NeRFSystem.setup                                                                                                                                                                          	|  31.037         	|  1              	|  31.037         	|  4.1075         	|
|  [TrainingEpochLoop].train_dataloader_next                                                                                                                                                                  	|  0.0015231      	|  20000          	|  30.462         	|  4.0315         	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                                                             	|  0.46207        	|  60             	|  27.724         	|  3.6691         	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                                                               	|  0.00044414     	|  20000          	|  8.8827         	|  1.1756         	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_end                 	|  0.14787        	|  20             	|  2.9574         	|  0.39139        	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                                                             	|  0.00011252     	|  20060          	|  2.2571         	|  0.29871        	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                                                                       	|  7.175e-05      	|  20060          	|  1.4393         	|  0.19048        	|
|  [EvaluationEpochLoop].val_dataloader_idx_0_next                                                                                                                                                            	|  0.0069227      	|  60             	|  0.41536        	|  0.054971       	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                                                            	|  1.6797e-05     	|  20000          	|  0.33593        	|  0.044459       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_end                 	|  1.3604e-05     	|  20000          	|  0.27209        	|  0.036009       	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                                                                    	|  1.1775e-05     	|  20000          	|  0.2355         	|  0.031167       	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                                                            	|  0.048777       	|  2              	|  0.097553       	|  0.012911       	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_end                                                                                                                                                 	|  4.1763e-06     	|  20000          	|  0.083526       	|  0.011054       	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                                                                	|  3.1929e-06     	|  20000          	|  0.063857       	|  0.0084511      	|
|  [Callback]TQDMProgressBar.on_batch_start                                                                                                                                                                   	|  2.9876e-06     	|  20000          	|  0.059752       	|  0.0079078      	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                                                                         	|  2.8599e-06     	|  20000          	|  0.057198       	|  0.0075698      	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                                                              	|  2.7543e-06     	|  20000          	|  0.055085       	|  0.0072902      	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                                                               	|  2.6779e-06     	|  20000          	|  0.053557       	|  0.007088       	|
|  [Callback]GradientAccumulationScheduler.on_batch_start                                                                                                                                                     	|  2.455e-06      	|  20000          	|  0.0491         	|  0.0064981      	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                                                             	|  2.4234e-06     	|  20000          	|  0.048468       	|  0.0064145      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_start                     	|  2.3846e-06     	|  20000          	|  0.047692       	|  0.0063117      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_start               	|  2.2444e-06     	|  20000          	|  0.044889       	|  0.0059407      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_after_backward                  	|  2.2231e-06     	|  20000          	|  0.044462       	|  0.0058843      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_start                                                                                                                                               	|  2.2061e-06     	|  20000          	|  0.044123       	|  0.0058394      	|
|  [Callback]TQDMProgressBar.on_batch_end                                                                                                                                                                     	|  2.0976e-06     	|  20000          	|  0.041953       	|  0.0055522      	|
|  [Callback]GradientAccumulationScheduler.on_before_optimizer_step                                                                                                                                           	|  2.0976e-06     	|  20000          	|  0.041952       	|  0.0055521      	|
|  [Callback]GradientAccumulationScheduler.on_after_backward                                                                                                                                                  	|  2.0625e-06     	|  20000          	|  0.041251       	|  0.0054593      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_optimizer_step           	|  2.047e-06      	|  20000          	|  0.04094        	|  0.0054182      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_end                       	|  2.0445e-06     	|  20000          	|  0.040891       	|  0.0054116      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_zero_grad                	|  2.0088e-06     	|  20000          	|  0.040177       	|  0.0053172      	|
|  [Callback]GradientAccumulationScheduler.on_before_zero_grad                                                                                                                                                	|  1.9826e-06     	|  20000          	|  0.039652       	|  0.0052477      	|
|  [Callback]GradientAccumulationScheduler.on_before_backward                                                                                                                                                 	|  1.9732e-06     	|  20000          	|  0.039463       	|  0.0052227      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_backward                 	|  1.9345e-06     	|  20000          	|  0.03869        	|  0.0051204      	|
|  [Callback]GradientAccumulationScheduler.on_batch_end                                                                                                                                                       	|  1.9203e-06     	|  20000          	|  0.038406       	|  0.0050828      	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                                                                          	|  0.0004719      	|  60             	|  0.028314       	|  0.0037472      	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                                                                       	|  1.108e-06      	|  20060          	|  0.022226       	|  0.0029415      	|
|  [LightningModule]NeRFSystem.training_step_end                                                                                                                                                              	|  1.008e-06      	|  20000          	|  0.020161       	|  0.0026681      	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                                                                        	|  9.6978e-07     	|  20060          	|  0.019454       	|  0.0025746      	|
|  [Strategy]SingleDeviceStrategy.training_step_end                                                                                                                                                           	|  9.653e-07      	|  20000          	|  0.019306       	|  0.002555       	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                                                             	|  9.391e-07      	|  20000          	|  0.018782       	|  0.0024857      	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                                                                           	|  8.596e-07      	|  20000          	|  0.017192       	|  0.0022752      	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                                                                        	|  7.7542e-07     	|  20000          	|  0.015508       	|  0.0020524      	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                                                                       	|  7.4595e-07     	|  20000          	|  0.014919       	|  0.0019744      	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                                                            	|  7.3882e-07     	|  20000          	|  0.014776       	|  0.0019556      	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                                                              	|  7.2663e-07     	|  20000          	|  0.014533       	|  0.0019233      	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                                                             	|  6.7665e-07     	|  20000          	|  0.013533       	|  0.001791       	|
|  [LightningModule]NeRFSystem.validation_epoch_end                                                                                                                                                           	|  0.0060249      	|  2              	|  0.01205        	|  0.0015947      	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                                                                           	|  0.0060946      	|  1              	|  0.0060946      	|  0.00080658     	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                                                               	|  0.00022123     	|  20             	|  0.0044247      	|  0.00058558     	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                                                             	|  0.00022087     	|  20             	|  0.0044174      	|  0.00058461     	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                                                                        	|  5.1732e-05     	|  60             	|  0.0031039      	|  0.00041078     	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                                                              	|  0.00010846     	|  20             	|  0.0021693      	|  0.00028709     	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                                                                       	|  0.00099056     	|  2              	|  0.0019811      	|  0.00026219     	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                                                                      	|  0.00070073     	|  2              	|  0.0014015      	|  0.00018548     	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                                                                	|  0.00061655     	|  2              	|  0.0012331      	|  0.00016319     	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                                                              	|  0.00059232     	|  2              	|  0.0011846      	|  0.00015678     	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                                                               	|  0.00058364     	|  1              	|  0.00058364     	|  7.7242e-05     	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                                                                 	|  0.00058149     	|  1              	|  0.00058149     	|  7.6957e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_end                                                                                                                                            	|  6.7461e-06     	|  60             	|  0.00040476     	|  5.3568e-05     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                                                                   	|  0.00029112     	|  1              	|  0.00029112     	|  3.8528e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_start                                                                                                                                          	|  4.5628e-06     	|  60             	|  0.00027377     	|  3.6232e-05     	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                                                                     	|  0.00026095     	|  1              	|  0.00026095     	|  3.4535e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_end            	|  3.3779e-06     	|  60             	|  0.00020267     	|  2.6823e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_start          	|  3.3233e-06     	|  60             	|  0.0001994      	|  2.6389e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_end                  	|  8.3974e-05     	|  2              	|  0.00016795     	|  2.2227e-05     	|
|  [LightningModule]NeRFSystem.validation_step_end                                                                                                                                                            	|  2.7341e-06     	|  60             	|  0.00016405     	|  2.1711e-05     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_start                                                                                                                                               	|  5.822e-06      	|  20             	|  0.00011644     	|  1.541e-05      	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                                                                      	|  1.8842e-06     	|  60             	|  0.00011305     	|  1.4961e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                                                                        	|  1.7455e-06     	|  60             	|  0.00010473     	|  1.3861e-05     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_end                                                                                                                                                 	|  4.3424e-06     	|  20             	|  8.6849e-05     	|  1.1494e-05     	|
|  [Strategy]SingleDeviceStrategy.validation_step_end                                                                                                                                                         	|  1.3952e-06     	|  60             	|  8.3709e-05     	|  1.1078e-05     	|
|  [Callback]TQDMProgressBar.on_epoch_start                                                                                                                                                                   	|  3.1355e-06     	|  22             	|  6.898e-05      	|  9.1291e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_end                                                                                                                                                                     	|  2.783e-06      	|  22             	|  6.1226e-05     	|  8.1029e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_start               	|  2.6651e-06     	|  20             	|  5.3301e-05     	|  7.0541e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_end                       	|  2.1136e-06     	|  22             	|  4.6498e-05     	|  6.1537e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_end                                                                                                                                                       	|  2.1104e-06     	|  22             	|  4.6429e-05     	|  6.1446e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_start                                                                                                                                                     	|  2.0677e-06     	|  22             	|  4.549e-05      	|  6.0203e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_start                     	|  2.0554e-06     	|  22             	|  4.5219e-05     	|  5.9844e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.setup                              	|  4.032e-05      	|  1              	|  4.032e-05      	|  5.3361e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                                                             	|  1.4805e-06     	|  20             	|  2.961e-05      	|  3.9186e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_start                                                                                                                                                                 	|  1.3367e-06     	|  22             	|  2.9408e-05     	|  3.892e-06      	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                                                                           	|  1.3775e-06     	|  20             	|  2.755e-05      	|  3.6461e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_end                                                                                                                                                                   	|  1.2277e-06     	|  22             	|  2.701e-05      	|  3.5746e-06     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                                                                        	|  1.156e-05      	|  2              	|  2.3119e-05     	|  3.0597e-06     	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                                                                           	|  1.118e-05      	|  2              	|  2.236e-05      	|  2.9592e-06     	|
|  [Callback]GradientAccumulationScheduler.on_validation_start                                                                                                                                                	|  5.9446e-06     	|  2              	|  1.1889e-05     	|  1.5735e-06     	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                                                            	|  1.014e-05      	|  1              	|  1.014e-05      	|  1.342e-06      	|
|  [Callback]GradientAccumulationScheduler.on_validation_end                                                                                                                                                  	|  4.7195e-06     	|  2              	|  9.439e-06      	|  1.2492e-06     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                                                                          	|  4.5998e-06     	|  2              	|  9.1996e-06     	|  1.2175e-06     	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                                                                     	|  8.7097e-06     	|  1              	|  8.7097e-06     	|  1.1527e-06     	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                                                            	|  7.9097e-06     	|  1              	|  7.9097e-06     	|  1.0468e-06     	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                                                                       	|  7.2308e-06     	|  1              	|  7.2308e-06     	|  9.5695e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                                                                        	|  3.4706e-06     	|  2              	|  6.9411e-06     	|  9.1862e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_start                                                                                                                                                        	|  6.55e-06       	|  1              	|  6.55e-06       	|  8.6685e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_start                                                                                                                                                       	|  6.3693e-06     	|  1              	|  6.3693e-06     	|  8.4294e-07     	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                                                                   	|  6.0704e-06     	|  1              	|  6.0704e-06     	|  8.0337e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_end                                                                                                                                            	|  2.8294e-06     	|  2              	|  5.6587e-06     	|  7.489e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_start                	|  2.7851e-06     	|  2              	|  5.5702e-06     	|  7.3719e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                                                                         	|  2.6999e-06     	|  2              	|  5.3998e-06     	|  7.1463e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_end            	|  2.6198e-06     	|  2              	|  5.2396e-06     	|  6.9343e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_start                                                                                                                                          	|  2.515e-06      	|  2              	|  5.0301e-06     	|  6.657e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_start          	|  2.4852e-06     	|  2              	|  4.9705e-06     	|  6.5781e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_start                                                                                                                                                     	|  4.7702e-06     	|  1              	|  4.7702e-06     	|  6.3131e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_start          	|  4.7395e-06     	|  1              	|  4.7395e-06     	|  6.2724e-07     	|
|  [LightningModule]NeRFSystem.configure_sharded_model                                                                                                                                                        	|  4.4797e-06     	|  1              	|  4.4797e-06     	|  5.9286e-07     	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                                                                   	|  4.34e-06       	|  1              	|  4.34e-06       	|  5.7437e-07     	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                                                                         	|  4.3204e-06     	|  1              	|  4.3204e-06     	|  5.7178e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_end                                                                                                                                                       	|  4.1705e-06     	|  1              	|  4.1705e-06     	|  5.5194e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_start                       	|  3.9199e-06     	|  1              	|  3.9199e-06     	|  5.1878e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_start                     	|  3.7905e-06     	|  1              	|  3.7905e-06     	|  5.0165e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_end                                                                                                                                                         	|  3.6899e-06     	|  1              	|  3.6899e-06     	|  4.8834e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_end                         	|  3.4804e-06     	|  1              	|  3.4804e-06     	|  4.606e-07      	|
|  [Callback]GradientAccumulationScheduler.teardown                                                                                                                                                           	|  3.3295e-06     	|  1              	|  3.3295e-06     	|  4.4064e-07     	|
|  [Callback]GradientAccumulationScheduler.setup                                                                                                                                                              	|  3.3202e-06     	|  1              	|  3.3202e-06     	|  4.394e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.teardown                           	|  3.1805e-06     	|  1              	|  3.1805e-06     	|  4.2092e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_start                                                                                                                                          	|  3.0901e-06     	|  1              	|  3.0901e-06     	|  4.0896e-07     	|
|  [Callback]TQDMProgressBar.on_configure_sharded_model                                                                                                                                                       	|  3.0305e-06     	|  1              	|  3.0305e-06     	|  4.0107e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                                                                      	|  1.485e-06      	|  2              	|  2.97e-06       	|  3.9306e-07     	|
|  [Callback]TQDMProgressBar.on_before_accelerator_backend_setup                                                                                                                                              	|  2.8694e-06     	|  1              	|  2.8694e-06     	|  3.7975e-07     	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                                                              	|  1.3048e-06     	|  2              	|  2.6096e-06     	|  3.4536e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_end                                                                                                                                                          	|  2.5295e-06     	|  1              	|  2.5295e-06     	|  3.3476e-07     	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                                                                   	|  2.4708e-06     	|  1              	|  2.4708e-06     	|  3.27e-07       	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_start                                                                                                                                                      	|  2.3805e-06     	|  1              	|  2.3805e-06     	|  3.1504e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_accelerator_backend_setup	|  2.17e-06       	|  1              	|  2.17e-06       	|  2.8718e-07     	|
|  [Callback]GradientAccumulationScheduler.on_configure_sharded_model                                                                                                                                         	|  2.1402e-06     	|  1              	|  2.1402e-06     	|  2.8324e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_end                       	|  2.1402e-06     	|  1              	|  2.1402e-06     	|  2.8324e-07     	|
|  [Callback]GradientAccumulationScheduler.on_before_accelerator_backend_setup                                                                                                                                	|  2.1094e-06     	|  1              	|  2.1094e-06     	|  2.7917e-07     	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                                                                       	|  2.0806e-06     	|  1              	|  2.0806e-06     	|  2.7535e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_configure_sharded_model         	|  1.9604e-06     	|  1              	|  1.9604e-06     	|  2.5945e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_end            	|  1.8803e-06     	|  1              	|  1.8803e-06     	|  2.4885e-07     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                                                                     	|  1.8803e-06     	|  1              	|  1.8803e-06     	|  2.4885e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_end                                                                                                                                            	|  1.8496e-06     	|  1              	|  1.8496e-06     	|  2.4478e-07     	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                                                             	|  1.5795e-06     	|  1              	|  1.5795e-06     	|  2.0904e-07     	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                                                                 	|  1.1502e-06     	|  1              	|  1.1502e-06     	|  1.5222e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_end                                                                                                                                                        	|  1.1399e-06     	|  1              	|  1.1399e-06     	|  1.5086e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                                                              	|  1.1297e-06     	|  1              	|  1.1297e-06     	|  1.4951e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                                                                	|  8.9034e-07     	|  1              	|  8.9034e-07     	|  1.1783e-07     	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
