FIT Profiler Report

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                                                                     	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                                                                      	|  -              	|  881073         	|  676.4          	|  100 %          	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                                                                         	|  32.694         	|  20             	|  653.88         	|  96.671         	|
|  run_training_batch                                                                                                                                                                                         	|  0.027846       	|  20000          	|  556.93         	|  82.337         	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                                                                 	|  0.027607       	|  20000          	|  552.15         	|  81.63          	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                                                               	|  0.009879       	|  20000          	|  197.58         	|  29.211         	|
|  [TrainingEpochLoop].train_dataloader_next                                                                                                                                                                  	|  0.0026108      	|  20000          	|  52.217         	|  7.7198         	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                                                                    	|  0.00183        	|  20000          	|  36.599         	|  5.4109         	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                                                             	|  0.58233        	|  38             	|  22.128         	|  3.2715         	|
|  [LightningModule]NeRFSystem.setup                                                                                                                                                                          	|  9.4902         	|  1              	|  9.4902         	|  1.403          	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                                                               	|  0.00046643     	|  20000          	|  9.3286         	|  1.3792         	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_end                 	|  0.16352        	|  20             	|  3.2704         	|  0.4835         	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                                                             	|  0.00011197     	|  20038          	|  2.2436         	|  0.3317         	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                                                                       	|  7.1944e-05     	|  20038          	|  1.4416         	|  0.21313        	|
|  [EvaluationEpochLoop].val_dataloader_idx_0_next                                                                                                                                                            	|  0.0099361      	|  38             	|  0.37757        	|  0.055821       	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                                                            	|  1.8101e-05     	|  20000          	|  0.36202        	|  0.053522       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_end                 	|  1.353e-05      	|  20000          	|  0.27059        	|  0.040005       	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                                                                    	|  1.2404e-05     	|  20000          	|  0.24809        	|  0.036678       	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_end                                                                                                                                                 	|  4.8337e-06     	|  20000          	|  0.096674       	|  0.014292       	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                                                            	|  0.04599        	|  2              	|  0.09198        	|  0.013598       	|
|  [Callback]TQDMProgressBar.on_batch_start                                                                                                                                                                   	|  3.146e-06      	|  20000          	|  0.062921       	|  0.0093023      	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                                                                	|  3.0295e-06     	|  20000          	|  0.060591       	|  0.0089579      	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                                                                         	|  2.9147e-06     	|  20000          	|  0.058294       	|  0.0086183      	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                                                              	|  2.7084e-06     	|  20000          	|  0.054168       	|  0.0080083      	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                                                               	|  2.651e-06      	|  20000          	|  0.053019       	|  0.0078384      	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                                                             	|  2.4299e-06     	|  20000          	|  0.048597       	|  0.0071847      	|
|  [Callback]GradientAccumulationScheduler.on_batch_start                                                                                                                                                     	|  2.4142e-06     	|  20000          	|  0.048284       	|  0.0071384      	|
|  [Callback]TQDMProgressBar.on_batch_end                                                                                                                                                                     	|  2.3686e-06     	|  20000          	|  0.047372       	|  0.0070036      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_start                     	|  2.327e-06      	|  20000          	|  0.046541       	|  0.0068807      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_start               	|  2.2207e-06     	|  20000          	|  0.044414       	|  0.0065662      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_start                                                                                                                                               	|  2.1795e-06     	|  20000          	|  0.04359        	|  0.0064445      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_after_backward                  	|  2.1236e-06     	|  20000          	|  0.042472       	|  0.0062791      	|
|  [Callback]GradientAccumulationScheduler.on_before_optimizer_step                                                                                                                                           	|  2.048e-06      	|  20000          	|  0.040961       	|  0.0060557      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_optimizer_step           	|  2.0424e-06     	|  20000          	|  0.040847       	|  0.006039       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_end                       	|  2.0421e-06     	|  20000          	|  0.040842       	|  0.0060381      	|
|  [Callback]GradientAccumulationScheduler.on_after_backward                                                                                                                                                  	|  2.0248e-06     	|  20000          	|  0.040496       	|  0.005987       	|
|  [Callback]GradientAccumulationScheduler.on_before_zero_grad                                                                                                                                                	|  2.0087e-06     	|  20000          	|  0.040174       	|  0.0059394      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_zero_grad                	|  1.9707e-06     	|  20000          	|  0.039415       	|  0.0058272      	|
|  [Callback]GradientAccumulationScheduler.on_batch_end                                                                                                                                                       	|  1.9406e-06     	|  20000          	|  0.038813       	|  0.0057382      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_backward                 	|  1.9387e-06     	|  20000          	|  0.038774       	|  0.0057325      	|
|  [Callback]GradientAccumulationScheduler.on_before_backward                                                                                                                                                 	|  1.9081e-06     	|  20000          	|  0.038162       	|  0.005642       	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                                                                       	|  1.0935e-06     	|  20038          	|  0.021913       	|  0.0032396      	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                                                             	|  1.052e-06      	|  20000          	|  0.02104        	|  0.0031106      	|
|  [LightningModule]NeRFSystem.training_step_end                                                                                                                                                              	|  1.0465e-06     	|  20000          	|  0.02093        	|  0.0030943      	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                                                                        	|  9.8571e-07     	|  20038          	|  0.019752       	|  0.0029201      	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                                                              	|  9.2166e-07     	|  20000          	|  0.018433       	|  0.0027252      	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                                                                          	|  0.00047978     	|  38             	|  0.018232       	|  0.0026954      	|
|  [Strategy]SingleDeviceStrategy.training_step_end                                                                                                                                                           	|  8.8652e-07     	|  20000          	|  0.01773        	|  0.0026213      	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                                                            	|  8.5932e-07     	|  20000          	|  0.017186       	|  0.0025409      	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                                                             	|  8.352e-07      	|  20000          	|  0.016704       	|  0.0024696      	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                                                                           	|  7.8308e-07     	|  20000          	|  0.015662       	|  0.0023154      	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                                                                        	|  7.812e-07      	|  20000          	|  0.015624       	|  0.0023099      	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                                                                       	|  7.5509e-07     	|  20000          	|  0.015102       	|  0.0022327      	|
|  [LightningModule]NeRFSystem.validation_epoch_end                                                                                                                                                           	|  0.0042714      	|  2              	|  0.0085428      	|  0.001263       	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                                                               	|  0.00023409     	|  20             	|  0.0046819      	|  0.00069218     	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                                                             	|  0.00022041     	|  20             	|  0.0044082      	|  0.00065172     	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                                                                           	|  0.003469       	|  1              	|  0.003469       	|  0.00051286     	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                                                                        	|  7.4368e-05     	|  38             	|  0.002826       	|  0.0004178      	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                                                              	|  9.3889e-05     	|  20             	|  0.0018778      	|  0.00027762     	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                                                                	|  0.00058175     	|  2              	|  0.0011635      	|  0.00017202     	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                                                                       	|  0.00055561     	|  2              	|  0.0011112      	|  0.00016429     	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                                                                      	|  0.00045297     	|  2              	|  0.00090593     	|  0.00013393     	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                                                              	|  0.00032927     	|  2              	|  0.00065853     	|  9.7359e-05     	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                                                               	|  0.00056091     	|  1              	|  0.00056091     	|  8.2927e-05     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                                                                   	|  0.00031134     	|  1              	|  0.00031134     	|  4.6029e-05     	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                                                                 	|  0.00027284     	|  1              	|  0.00027284     	|  4.0337e-05     	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                                                                     	|  0.00025945     	|  1              	|  0.00025945     	|  3.8357e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_end                                                                                                                                            	|  6.1433e-06     	|  38             	|  0.00023345     	|  3.4513e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_start                                                                                                                                          	|  3.8652e-06     	|  38             	|  0.00014688     	|  2.1715e-05     	|
|  [LightningModule]NeRFSystem.validation_step_end                                                                                                                                                            	|  3.5655e-06     	|  38             	|  0.00013549     	|  2.0031e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_end            	|  3.0118e-06     	|  38             	|  0.00011445     	|  1.692e-05      	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_start                                                                                                                                               	|  5.4679e-06     	|  20             	|  0.00010936     	|  1.6168e-05     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_end                                                                                                                                                 	|  5.0544e-06     	|  20             	|  0.00010109     	|  1.4945e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_start          	|  2.6555e-06     	|  38             	|  0.00010091     	|  1.4919e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_end                  	|  3.7815e-05     	|  2              	|  7.5631e-05     	|  1.1181e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                                                                        	|  1.9808e-06     	|  38             	|  7.5272e-05     	|  1.1128e-05     	|
|  [Callback]TQDMProgressBar.on_epoch_start                                                                                                                                                                   	|  3.1027e-06     	|  22             	|  6.8259e-05     	|  1.0092e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                                                                      	|  1.7161e-06     	|  38             	|  6.5211e-05     	|  9.641e-06      	|
|  [Callback]TQDMProgressBar.on_epoch_end                                                                                                                                                                     	|  2.9496e-06     	|  22             	|  6.4891e-05     	|  9.5936e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_start               	|  2.7444e-06     	|  20             	|  5.4888e-05     	|  8.1148e-06     	|
|  [Strategy]SingleDeviceStrategy.validation_step_end                                                                                                                                                         	|  1.3149e-06     	|  38             	|  4.9967e-05     	|  7.3873e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_end                                                                                                                                                       	|  2.0328e-06     	|  22             	|  4.4722e-05     	|  6.6118e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_end                       	|  2.0178e-06     	|  22             	|  4.4391e-05     	|  6.5629e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_start                     	|  1.9796e-06     	|  22             	|  4.3551e-05     	|  6.4386e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_start                                                                                                                                                     	|  1.9748e-06     	|  22             	|  4.3446e-05     	|  6.4232e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.setup                              	|  3.304e-05      	|  1              	|  3.304e-05      	|  4.8846e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_end                                                                                                                                                                   	|  1.3136e-06     	|  22             	|  2.89e-05       	|  4.2726e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                                                                           	|  1.4189e-06     	|  20             	|  2.8378e-05     	|  4.1955e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                                                             	|  1.379e-06      	|  20             	|  2.758e-05      	|  4.0775e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_start                                                                                                                                                                 	|  1.2401e-06     	|  22             	|  2.7282e-05     	|  4.0335e-06     	|
|  [Callback]GradientAccumulationScheduler.on_validation_start                                                                                                                                                	|  5.8897e-06     	|  2              	|  1.1779e-05     	|  1.7415e-06     	|
|  [Callback]GradientAccumulationScheduler.on_validation_end                                                                                                                                                  	|  5.4804e-06     	|  2              	|  1.0961e-05     	|  1.6205e-06     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                                                                          	|  4.9248e-06     	|  2              	|  9.8497e-06     	|  1.4562e-06     	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                                                            	|  8.3102e-06     	|  1              	|  8.3102e-06     	|  1.2286e-06     	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                                                                       	|  8.1696e-06     	|  1              	|  8.1696e-06     	|  1.2078e-06     	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                                                                           	|  3.5004e-06     	|  2              	|  7.0008e-06     	|  1.035e-06      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_start                	|  3.2852e-06     	|  2              	|  6.5705e-06     	|  9.7139e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_end                                                                                                                                            	|  3.0799e-06     	|  2              	|  6.1598e-06     	|  9.1067e-07     	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                                                                   	|  6.0694e-06     	|  1              	|  6.0694e-06     	|  8.9732e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_end            	|  3.0147e-06     	|  2              	|  6.0294e-06     	|  8.914e-07      	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                                                            	|  5.669e-06      	|  1              	|  5.669e-06      	|  8.3811e-07     	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                                                                         	|  5.1502e-06     	|  1              	|  5.1502e-06     	|  7.6142e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                                                                        	|  2.5355e-06     	|  2              	|  5.0711e-06     	|  7.4971e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_start                                                                                                                                                     	|  5.0096e-06     	|  1              	|  5.0096e-06     	|  7.4063e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_start                     	|  4.8401e-06     	|  1              	|  4.8401e-06     	|  7.1557e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_end                                                                                                                                                         	|  4.4098e-06     	|  1              	|  4.4098e-06     	|  6.5196e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_end                                                                                                                                                       	|  4.4005e-06     	|  1              	|  4.4005e-06     	|  6.5058e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_start                                                                                                                                                        	|  4.2599e-06     	|  1              	|  4.2599e-06     	|  6.2979e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_end                         	|  3.8901e-06     	|  1              	|  3.8901e-06     	|  5.7513e-07     	|
|  [Callback]GradientAccumulationScheduler.teardown                                                                                                                                                           	|  3.8492e-06     	|  1              	|  3.8492e-06     	|  5.6907e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_start          	|  1.8952e-06     	|  2              	|  3.7905e-06     	|  5.6039e-07     	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                                                                     	|  3.7402e-06     	|  1              	|  3.7402e-06     	|  5.5296e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.teardown                           	|  3.7402e-06     	|  1              	|  3.7402e-06     	|  5.5296e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_start                                                                                                                                          	|  1.8552e-06     	|  2              	|  3.7104e-06     	|  5.4855e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                                                                      	|  1.7947e-06     	|  2              	|  3.5893e-06     	|  5.3065e-07     	|
|  [Callback]TQDMProgressBar.on_configure_sharded_model                                                                                                                                                       	|  3.2997e-06     	|  1              	|  3.2997e-06     	|  4.8783e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                                                                        	|  1.6298e-06     	|  2              	|  3.2596e-06     	|  4.8191e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                                                                         	|  1.58e-06       	|  2              	|  3.16e-06       	|  4.6718e-07     	|
|  [LightningModule]NeRFSystem.configure_sharded_model                                                                                                                                                        	|  3.0501e-06     	|  1              	|  3.0501e-06     	|  4.5093e-07     	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                                                             	|  2.8601e-06     	|  1              	|  2.8601e-06     	|  4.2284e-07     	|
|  [Callback]TQDMProgressBar.on_before_accelerator_backend_setup                                                                                                                                              	|  2.8592e-06     	|  1              	|  2.8592e-06     	|  4.227e-07      	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                                                                       	|  2.8005e-06     	|  1              	|  2.8005e-06     	|  4.1403e-07     	|
|  [Callback]GradientAccumulationScheduler.setup                                                                                                                                                              	|  2.5602e-06     	|  1              	|  2.5602e-06     	|  3.7851e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_end                                                                                                                                                          	|  2.5099e-06     	|  1              	|  2.5099e-06     	|  3.7107e-07     	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                                                              	|  1.2051e-06     	|  2              	|  2.4103e-06     	|  3.5634e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_start                                                                                                                                                       	|  2.4093e-06     	|  1              	|  2.4093e-06     	|  3.562e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_start          	|  2.2193e-06     	|  1              	|  2.2193e-06     	|  3.2811e-07     	|
|  [Callback]GradientAccumulationScheduler.on_before_accelerator_backend_setup                                                                                                                                	|  2.21e-06       	|  1              	|  2.21e-06       	|  3.2674e-07     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                                                                     	|  2.1998e-06     	|  1              	|  2.1998e-06     	|  3.2522e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_accelerator_backend_setup	|  2.17e-06       	|  1              	|  2.17e-06       	|  3.2081e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_start                                                                                                                                          	|  2.1597e-06     	|  1              	|  2.1597e-06     	|  3.193e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_end                       	|  2.1495e-06     	|  1              	|  2.1495e-06     	|  3.1779e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_start                       	|  2.0899e-06     	|  1              	|  2.0899e-06     	|  3.0897e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_configure_sharded_model         	|  2.0107e-06     	|  1              	|  2.0107e-06     	|  2.9727e-07     	|
|  [Callback]GradientAccumulationScheduler.on_configure_sharded_model                                                                                                                                         	|  1.98e-06       	|  1              	|  1.98e-06       	|  2.9273e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_end            	|  1.8794e-06     	|  1              	|  1.8794e-06     	|  2.7786e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_end                                                                                                                                            	|  1.8701e-06     	|  1              	|  1.8701e-06     	|  2.7648e-07     	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                                                                   	|  1.8403e-06     	|  1              	|  1.8403e-06     	|  2.7207e-07     	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                                                                   	|  1.83e-06       	|  1              	|  1.83e-06       	|  2.7056e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_start                                                                                                                                                      	|  1.7006e-06     	|  1              	|  1.7006e-06     	|  2.5142e-07     	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                                                                 	|  1.5898e-06     	|  1              	|  1.5898e-06     	|  2.3503e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_end                                                                                                                                                        	|  1.3188e-06     	|  1              	|  1.3188e-06     	|  1.9497e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                                                                	|  1.2796e-06     	|  1              	|  1.2796e-06     	|  1.8918e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                                                              	|  1.0794e-06     	|  1              	|  1.0794e-06     	|  1.5958e-07     	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
