FIT Profiler Report

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                                                                     	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                                                                      	|  -              	|  881713         	|  1454.3         	|  100 %          	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                                                                         	|  70.371         	|  20             	|  1407.4         	|  96.777         	|
|  run_training_batch                                                                                                                                                                                         	|  0.059581       	|  20000          	|  1191.6         	|  81.937         	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                                                                 	|  0.059128       	|  20000          	|  1182.6         	|  81.315         	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                                                               	|  0.021096       	|  20000          	|  421.92         	|  29.012         	|
|  [TrainingEpochLoop].train_dataloader_next                                                                                                                                                                  	|  0.0071889      	|  20000          	|  143.78         	|  9.8864         	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                                                                    	|  0.0051619      	|  20000          	|  103.24         	|  7.0989         	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                                                             	|  0.54089        	|  78             	|  42.189         	|  2.901          	|
|  [LightningModule]NeRFSystem.setup                                                                                                                                                                          	|  35.018         	|  1              	|  35.018         	|  2.4079         	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                                                               	|  0.00055566     	|  20000          	|  11.113         	|  0.76416        	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_end                 	|  0.22721        	|  20             	|  4.5442         	|  0.31247        	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                                                             	|  0.00018207     	|  20078          	|  3.6557         	|  0.25137        	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                                                                       	|  0.0001186      	|  20078          	|  2.3812         	|  0.16374        	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                                                            	|  3.5826e-05     	|  20000          	|  0.71653        	|  0.049269       	|
|  [EvaluationEpochLoop].val_dataloader_idx_0_next                                                                                                                                                            	|  0.0061304      	|  78             	|  0.47817        	|  0.03288        	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                                                                    	|  1.9397e-05     	|  20000          	|  0.38793        	|  0.026675       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_end                 	|  1.8109e-05     	|  20000          	|  0.36218        	|  0.024904       	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                                                            	|  0.054562       	|  2              	|  0.10912        	|  0.0075036      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_end                                                                                                                                                 	|  5.1814e-06     	|  20000          	|  0.10363        	|  0.0071257      	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                                                                	|  4.3122e-06     	|  20000          	|  0.086243       	|  0.0059302      	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                                                                         	|  3.7192e-06     	|  20000          	|  0.074384       	|  0.0051148      	|
|  [Callback]TQDMProgressBar.on_batch_start                                                                                                                                                                   	|  3.7084e-06     	|  20000          	|  0.074169       	|  0.0051         	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                                                             	|  3.3327e-06     	|  20000          	|  0.066654       	|  0.0045832      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_start               	|  3.3e-06        	|  20000          	|  0.065999       	|  0.0045382      	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                                                              	|  3.206e-06      	|  20000          	|  0.06412        	|  0.004409       	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                                                               	|  3.1155e-06     	|  20000          	|  0.06231        	|  0.0042845      	|
|  [Callback]GradientAccumulationScheduler.on_batch_start                                                                                                                                                     	|  3.0176e-06     	|  20000          	|  0.060352       	|  0.0041499      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_start                     	|  2.9423e-06     	|  20000          	|  0.058845       	|  0.0040463      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_start                                                                                                                                               	|  2.7018e-06     	|  20000          	|  0.054036       	|  0.0037156      	|
|  [Callback]TQDMProgressBar.on_batch_end                                                                                                                                                                     	|  2.6465e-06     	|  20000          	|  0.05293        	|  0.0036395      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_after_backward                  	|  2.4655e-06     	|  20000          	|  0.04931        	|  0.0033907      	|
|  [Callback]GradientAccumulationScheduler.on_before_optimizer_step                                                                                                                                           	|  2.4013e-06     	|  20000          	|  0.048026       	|  0.0033024      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_optimizer_step           	|  2.3553e-06     	|  20000          	|  0.047105       	|  0.003239       	|
|  [Callback]GradientAccumulationScheduler.on_after_backward                                                                                                                                                  	|  2.346e-06      	|  20000          	|  0.04692        	|  0.0032263      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_end                       	|  2.2671e-06     	|  20000          	|  0.045342       	|  0.0031178      	|
|  [Callback]GradientAccumulationScheduler.on_batch_end                                                                                                                                                       	|  2.2566e-06     	|  20000          	|  0.045133       	|  0.0031034      	|
|  [Callback]GradientAccumulationScheduler.on_before_zero_grad                                                                                                                                                	|  2.2124e-06     	|  20000          	|  0.044248       	|  0.0030426      	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                                                                       	|  2.1677e-06     	|  20078          	|  0.043524       	|  0.0029928      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_zero_grad                	|  2.1622e-06     	|  20000          	|  0.043245       	|  0.0029736      	|
|  [Callback]GradientAccumulationScheduler.on_before_backward                                                                                                                                                 	|  2.0724e-06     	|  20000          	|  0.041447       	|  0.00285        	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_backward                 	|  2.0711e-06     	|  20000          	|  0.041422       	|  0.0028482      	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                                                                          	|  0.00049465     	|  78             	|  0.038583       	|  0.002653       	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                                                                        	|  1.5222e-06     	|  20078          	|  0.030563       	|  0.0021015      	|
|  [LightningModule]NeRFSystem.training_step_end                                                                                                                                                              	|  1.3588e-06     	|  20000          	|  0.027176       	|  0.0018686      	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                                                             	|  1.333e-06      	|  20000          	|  0.02666        	|  0.0018332      	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                                                                           	|  1.239e-06      	|  20000          	|  0.024779       	|  0.0017039      	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                                                              	|  1.2244e-06     	|  20000          	|  0.024489       	|  0.0016839      	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                                                                       	|  1.1794e-06     	|  20000          	|  0.023588       	|  0.001622       	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                                                                        	|  1.1507e-06     	|  20000          	|  0.023014       	|  0.0015825      	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                                                            	|  1.1397e-06     	|  20000          	|  0.022793       	|  0.0015673      	|
|  [Strategy]SingleDeviceStrategy.training_step_end                                                                                                                                                           	|  1.1395e-06     	|  20000          	|  0.02279        	|  0.0015671      	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                                                             	|  1.0794e-06     	|  20000          	|  0.021589       	|  0.0014845      	|
|  [LightningModule]NeRFSystem.validation_epoch_end                                                                                                                                                           	|  0.0051937      	|  2              	|  0.010387       	|  0.00071425     	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                                                               	|  0.00028888     	|  20             	|  0.0057775      	|  0.00039727     	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                                                             	|  0.00023962     	|  20             	|  0.0047924      	|  0.00032953     	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                                                                           	|  0.0038832      	|  1              	|  0.0038832      	|  0.00026702     	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                                                                        	|  2.9613e-05     	|  78             	|  0.0023098      	|  0.00015883     	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                                                              	|  0.00010236     	|  20             	|  0.0020471      	|  0.00014076     	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                                                                	|  0.00070428     	|  2              	|  0.0014086      	|  9.6855e-05     	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                                                                       	|  0.00047471     	|  2              	|  0.00094942     	|  6.5284e-05     	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                                                                      	|  0.00042823     	|  2              	|  0.00085645     	|  5.8891e-05     	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                                                               	|  0.00079856     	|  1              	|  0.00079856     	|  5.491e-05      	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_end                                                                                                                                            	|  6.7986e-06     	|  78             	|  0.00053029     	|  3.6463e-05     	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                                                              	|  0.00024373     	|  2              	|  0.00048747     	|  3.3519e-05     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                                                                   	|  0.00040335     	|  1              	|  0.00040335     	|  2.7735e-05     	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                                                                     	|  0.00030964     	|  1              	|  0.00030964     	|  2.1291e-05     	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                                                                 	|  0.00027796     	|  1              	|  0.00027796     	|  1.9113e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_start                                                                                                                                          	|  3.5585e-06     	|  78             	|  0.00027757     	|  1.9086e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_end            	|  3.3686e-06     	|  78             	|  0.00026275     	|  1.8067e-05     	|
|  [LightningModule]NeRFSystem.validation_step_end                                                                                                                                                            	|  3.3619e-06     	|  78             	|  0.00026223     	|  1.8031e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_start          	|  2.8961e-06     	|  78             	|  0.0002259      	|  1.5533e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                                                                        	|  2.3259e-06     	|  78             	|  0.00018142     	|  1.2475e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                                                                      	|  1.4947e-06     	|  78             	|  0.00011659     	|  8.0168e-06     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_start                                                                                                                                               	|  5.7274e-06     	|  20             	|  0.00011455     	|  7.8766e-06     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_end                                                                                                                                                 	|  4.9075e-06     	|  20             	|  9.815e-05      	|  6.749e-06      	|
|  [Strategy]SingleDeviceStrategy.validation_step_end                                                                                                                                                         	|  1.2543e-06     	|  78             	|  9.7832e-05     	|  6.7271e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_start                                                                                                                                                                   	|  3.5117e-06     	|  22             	|  7.7258e-05     	|  5.3124e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_end                  	|  3.5224e-05     	|  2              	|  7.0448e-05     	|  4.8441e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_end                                                                                                                                                                     	|  3.1114e-06     	|  22             	|  6.845e-05      	|  4.7068e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_start               	|  2.9245e-06     	|  20             	|  5.8491e-05     	|  4.0219e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_start                                                                                                                                                     	|  2.2218e-06     	|  22             	|  4.888e-05      	|  3.361e-06      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_start                     	|  2.2124e-06     	|  22             	|  4.8674e-05     	|  3.3469e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_end                       	|  2.2014e-06     	|  22             	|  4.843e-05      	|  3.3301e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_end                                                                                                                                                       	|  2.1955e-06     	|  22             	|  4.8301e-05     	|  3.3213e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.setup                              	|  4.3719e-05     	|  1              	|  4.3719e-05     	|  3.0062e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_start                                                                                                                                                                 	|  1.4905e-06     	|  22             	|  3.2792e-05     	|  2.2548e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                                                             	|  1.5845e-06     	|  20             	|  3.169e-05      	|  2.1791e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_end                                                                                                                                                                   	|  1.39e-06       	|  22             	|  3.0581e-05     	|  2.1028e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                                                                           	|  1.462e-06      	|  20             	|  2.9241e-05     	|  2.0106e-06     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                                                                          	|  1.063e-05      	|  2              	|  2.1259e-05     	|  1.4618e-06     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                                                                        	|  7.6103e-06     	|  2              	|  1.5221e-05     	|  1.0466e-06     	|
|  [Callback]GradientAccumulationScheduler.on_validation_end                                                                                                                                                  	|  4.6953e-06     	|  2              	|  9.3905e-06     	|  6.4571e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_start                                                                                                                                                	|  4.3698e-06     	|  2              	|  8.7395e-06     	|  6.0094e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_end                                                                                                                                            	|  3.5502e-06     	|  2              	|  7.1004e-06     	|  4.8823e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_start                                                                                                                                                        	|  6.8899e-06     	|  1              	|  6.8899e-06     	|  4.7376e-07     	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                                                            	|  6.7502e-06     	|  1              	|  6.7502e-06     	|  4.6416e-07     	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                                                            	|  6.6906e-06     	|  1              	|  6.6906e-06     	|  4.6006e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_end            	|  3.3397e-06     	|  2              	|  6.6794e-06     	|  4.5929e-07     	|
|  [LightningModule]NeRFSystem.configure_sharded_model                                                                                                                                                        	|  6.6701e-06     	|  1              	|  6.6701e-06     	|  4.5865e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                                                                           	|  3.1651e-06     	|  2              	|  6.3302e-06     	|  4.3527e-07     	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                                                                       	|  6.0098e-06     	|  1              	|  6.0098e-06     	|  4.1324e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                                                                        	|  2.9597e-06     	|  2              	|  5.9195e-06     	|  4.0703e-07     	|
|  [Callback]TQDMProgressBar.on_configure_sharded_model                                                                                                                                                       	|  5.3402e-06     	|  1              	|  5.3402e-06     	|  3.672e-07      	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                                                                     	|  5.2899e-06     	|  1              	|  5.2899e-06     	|  3.6374e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_start                                                                                                                                                     	|  5.0003e-06     	|  1              	|  5.0003e-06     	|  3.4383e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_start          	|  2.2999e-06     	|  2              	|  4.5998e-06     	|  3.1629e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_start                                                                                                                                          	|  2.2599e-06     	|  2              	|  4.5197e-06     	|  3.1078e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_end                                                                                                                                                       	|  4.4703e-06     	|  1              	|  4.4703e-06     	|  3.0739e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_start                     	|  4.4303e-06     	|  1              	|  4.4303e-06     	|  3.0463e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_start                	|  2.0247e-06     	|  2              	|  4.0494e-06     	|  2.7844e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                                                                         	|  1.8999e-06     	|  2              	|  3.7998e-06     	|  2.6128e-07     	|
|  [Callback]GradientAccumulationScheduler.on_configure_sharded_model                                                                                                                                         	|  3.6396e-06     	|  1              	|  3.6396e-06     	|  2.5027e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_configure_sharded_model         	|  3.4701e-06     	|  1              	|  3.4701e-06     	|  2.3861e-07     	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                                                                   	|  3.2103e-06     	|  1              	|  3.2103e-06     	|  2.2074e-07     	|
|  [Callback]TQDMProgressBar.on_before_accelerator_backend_setup                                                                                                                                              	|  3.0799e-06     	|  1              	|  3.0799e-06     	|  2.1178e-07     	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                                                              	|  1.5297e-06     	|  2              	|  3.0594e-06     	|  2.1037e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                                                                      	|  1.4799e-06     	|  2              	|  2.9597e-06     	|  2.0352e-07     	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                                                                         	|  2.8498e-06     	|  1              	|  2.8498e-06     	|  1.9596e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_end                                                                                                                                                          	|  2.7902e-06     	|  1              	|  2.7902e-06     	|  1.9186e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                                                              	|  2.74e-06       	|  1              	|  2.74e-06       	|  1.884e-07      	|
|  [Callback]GradientAccumulationScheduler.on_fit_start                                                                                                                                                       	|  2.7101e-06     	|  1              	|  2.7101e-06     	|  1.8635e-07     	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                                                                   	|  2.6701e-06     	|  1              	|  2.6701e-06     	|  1.836e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_end                       	|  2.6701e-06     	|  1              	|  2.6701e-06     	|  1.836e-07      	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_end                                                                                                                                                        	|  2.5798e-06     	|  1              	|  2.5798e-06     	|  1.7739e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_start                                                                                                                                                      	|  2.5705e-06     	|  1              	|  2.5705e-06     	|  1.7675e-07     	|
|  [Callback]GradientAccumulationScheduler.setup                                                                                                                                                              	|  2.4494e-06     	|  1              	|  2.4494e-06     	|  1.6842e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_end                                                                                                                                                         	|  2.4205e-06     	|  1              	|  2.4205e-06     	|  1.6644e-07     	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                                                             	|  2.3395e-06     	|  1              	|  2.3395e-06     	|  1.6087e-07     	|
|  [Callback]GradientAccumulationScheduler.on_before_accelerator_backend_setup                                                                                                                                	|  2.1895e-06     	|  1              	|  2.1895e-06     	|  1.5056e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_start                       	|  2.1597e-06     	|  1              	|  2.1597e-06     	|  1.4851e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_start                                                                                                                                          	|  2.1597e-06     	|  1              	|  2.1597e-06     	|  1.4851e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_start          	|  2.1392e-06     	|  1              	|  2.1392e-06     	|  1.471e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_end                         	|  2.1197e-06     	|  1              	|  2.1197e-06     	|  1.4575e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_accelerator_backend_setup	|  2.0498e-06     	|  1              	|  2.0498e-06     	|  1.4095e-07     	|
|  [Callback]GradientAccumulationScheduler.teardown                                                                                                                                                           	|  2.0396e-06     	|  1              	|  2.0396e-06     	|  1.4025e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_end            	|  1.9399e-06     	|  1              	|  1.9399e-06     	|  1.3339e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.teardown                           	|  1.8897e-06     	|  1              	|  1.8897e-06     	|  1.2994e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_end                                                                                                                                            	|  1.8794e-06     	|  1              	|  1.8794e-06     	|  1.2923e-07     	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                                                                   	|  1.7704e-06     	|  1              	|  1.7704e-06     	|  1.2174e-07     	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                                                                       	|  1.5302e-06     	|  1              	|  1.5302e-06     	|  1.0522e-07     	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                                                                 	|  1.4193e-06     	|  1              	|  1.4193e-06     	|  9.7596e-08     	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                                                                	|  1.3802e-06     	|  1              	|  1.3802e-06     	|  9.4906e-08     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                                                                     	|  1.3094e-06     	|  1              	|  1.3094e-06     	|  9.0039e-08     	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
