Watch Question
Training Stage getting an error
Related to: AI Unlimited
Date: 09/12/2022 10:22
User: Neves Chrichlow
Awards:
User: Neves Chrichlow
Awards:
An error keeps coming up here. Also there is no Xformers after Stage 2 dependencies
########:'########:::::'###::::'####:'##::: ##:'####:'##::: ##::'######:::
... ##..:: ##.... ##:::'## ##:::. ##:: ###:: ##:. ##:: ###:: ##:'##... ##::
::: ##:::: ##:::: ##::'##:. ##::: ##:: ####: ##:: ##:: ####: ##: ##:::..:::
::: ##:::: ########::'##:::. ##:: ##:: ## ## ##:: ##:: ## ## ##: ##::'####:
::: ##:::: ##.. ##::: #########:: ##:: ##. ####:: ##:: ##. ####: ##::: ##::
::: ##:::: ##::. ##:: ##.... ##:: ##:: ##:. ###:: ##:: ##:. ###: ##::: ##::
::: ##:::: ##:::. ##: ##:::: ##:'####: ##::. ##:'####: ##::. ##:. ######:::
:::..:::::..:::::..::..:::::..::....::..::::..::....::..::::..:::......::::
Progress:| | 1% 11/1600 [00:20<36:34, 1.38s/it, loss=0.0787, lr=1e-6]Traceback (most recent call last):
File "/content/diffusers/examples/dreambooth/train_dreambooth.py", line 797, in
main()
File "/content/diffusers/examples/dreambooth/train_dreambooth.py", line 632, in main
for step, batch in enumerate(train_dataloader):
File "/usr/local/lib/python3.8/dist-packages/accelerate/data_loader.py", line 357, in __iter__
next_batch = next(dataloader_iter)
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/dataloader.py", line 681, in __next__
data = self._next_data()
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/dataloader.py", line 721, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/fetch.py", line 49, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/content/diffusers/examples/dreambooth/train_dreambooth.py", line 324, in __getitem__
instance_image = Image.open(path)
File "/usr/local/lib/python3.8/dist-packages/PIL/Image.py", line 3131, in open
fp = builtins.open(filename, "rb")
IsADirectoryError: [Errno 21] Is a directory: '/content/data/nevesrich1/.ipynb_checkpoints'
Progress:| | 1% 11/1600 [00:21<50:47, 1.92s/it, loss=0.0787, lr=1e-6]
Traceback (most recent call last):
File "/usr/local/bin/accelerate", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.8/dist-packages/accelerate/commands/accelerate_cli.py", line 43, in main
args.func(args)
File "/usr/local/lib/python3.8/dist-packages/accelerate/commands/launch.py", line 837, in launch_command
simple_launcher(args)
File "/usr/local/lib/python3.8/dist-packages/accelerate/commands/launch.py", line 354, in simple_launcher
raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['/usr/bin/python3', '/content/diffusers/examples/dreambooth/train_dreambooth.py', '--save_starting_step=500', '--save_n_steps=0', '--train_text_encoder', '--pretrained_model_name_or_path=/content/stable-diffusion-v1-5', '--instance_data_dir=/content/data/nevesrich1', '--class_data_dir=/content/regularization_images/person_ddim', '--output_dir=/content/models/nevesrich1', '--with_prior_preservation', '--prior_loss_weight=1.0', '--instance_prompt=photo of nevesrich1 person', '--class_prompt=a photo of a person, ultra detailed', '--seed=75576', '--resolution=512', '--mixed_precision=fp16', '--train_batch_size=1', '--gradient_accumulation_steps=1', '--gradient_checkpointing', '--use_8bit_adam', '--learning_rate=1e-6', '--lr_scheduler=constant', '--lr_warmup_steps=0', '--center_crop', '--max_train_steps=1600', '--num_class_images=500']' returned non-zero exit status 1.
Something went wrong
########:'########:::::'###::::'####:'##::: ##:'####:'##::: ##::'######:::
... ##..:: ##.... ##:::'## ##:::. ##:: ###:: ##:. ##:: ###:: ##:'##... ##::
::: ##:::: ##:::: ##::'##:. ##::: ##:: ####: ##:: ##:: ####: ##: ##:::..:::
::: ##:::: ########::'##:::. ##:: ##:: ## ## ##:: ##:: ## ## ##: ##::'####:
::: ##:::: ##.. ##::: #########:: ##:: ##. ####:: ##:: ##. ####: ##::: ##::
::: ##:::: ##::. ##:: ##.... ##:: ##:: ##:. ###:: ##:: ##:. ###: ##::: ##::
::: ##:::: ##:::. ##: ##:::: ##:'####: ##::. ##:'####: ##::. ##:. ######:::
:::..:::::..:::::..::..:::::..::....::..::::..::....::..::::..:::......::::
Progress:| | 1% 11/1600 [00:20<36:34, 1.38s/it, loss=0.0787, lr=1e-6]Traceback (most recent call last):
File "/content/diffusers/examples/dreambooth/train_dreambooth.py", line 797, in
main()
File "/content/diffusers/examples/dreambooth/train_dreambooth.py", line 632, in main
for step, batch in enumerate(train_dataloader):
File "/usr/local/lib/python3.8/dist-packages/accelerate/data_loader.py", line 357, in __iter__
next_batch = next(dataloader_iter)
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/dataloader.py", line 681, in __next__
data = self._next_data()
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/dataloader.py", line 721, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/fetch.py", line 49, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/content/diffusers/examples/dreambooth/train_dreambooth.py", line 324, in __getitem__
instance_image = Image.open(path)
File "/usr/local/lib/python3.8/dist-packages/PIL/Image.py", line 3131, in open
fp = builtins.open(filename, "rb")
IsADirectoryError: [Errno 21] Is a directory: '/content/data/nevesrich1/.ipynb_checkpoints'
Progress:| | 1% 11/1600 [00:21<50:47, 1.92s/it, loss=0.0787, lr=1e-6]
Traceback (most recent call last):
File "/usr/local/bin/accelerate", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.8/dist-packages/accelerate/commands/accelerate_cli.py", line 43, in main
args.func(args)
File "/usr/local/lib/python3.8/dist-packages/accelerate/commands/launch.py", line 837, in launch_command
simple_launcher(args)
File "/usr/local/lib/python3.8/dist-packages/accelerate/commands/launch.py", line 354, in simple_launcher
raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['/usr/bin/python3', '/content/diffusers/examples/dreambooth/train_dreambooth.py', '--save_starting_step=500', '--save_n_steps=0', '--train_text_encoder', '--pretrained_model_name_or_path=/content/stable-diffusion-v1-5', '--instance_data_dir=/content/data/nevesrich1', '--class_data_dir=/content/regularization_images/person_ddim', '--output_dir=/content/models/nevesrich1', '--with_prior_preservation', '--prior_loss_weight=1.0', '--instance_prompt=photo of nevesrich1 person', '--class_prompt=a photo of a person, ultra detailed', '--seed=75576', '--resolution=512', '--mixed_precision=fp16', '--train_batch_size=1', '--gradient_accumulation_steps=1', '--gradient_checkpointing', '--use_8bit_adam', '--learning_rate=1e-6', '--lr_scheduler=constant', '--lr_warmup_steps=0', '--center_crop', '--max_train_steps=1600', '--num_class_images=500']' returned non-zero exit status 1.
Something went wrong
Answers
Date: 04/12/2022 21:30
User: Jorge Vila
Awards:
User: Jorge Vila
Awards:
No need for xformers anymore.
Have you deleted SD folder?
Doi you have enough space?
Otherwise hit me up at jorge@jorgevila.com and we will setup a Zoom call so I can helpo you out!
Cheers
Have you deleted SD folder?
Doi you have enough space?
Otherwise hit me up at jorge@jorgevila.com and we will setup a Zoom call so I can helpo you out!
Cheers
Date: 05/12/2022 01:33
User: Neves Chrichlow
Awards:
User: Neves Chrichlow
Awards:
No I have not deleted the SD folder that was initially set up. If I delete would this not delete the set up in the first video
Date: 05/12/2022 01:42
User: Neves Chrichlow
Awards:
User: Neves Chrichlow
Awards:
In the video for " Create Ai Images" you indicated that any SD folder should be deleted but this is where the files were uploaded to. In the video for creating your own image
do you want me to delete this folder with the files from the " Create AI Images" and start over with a new folder now? Wouldn't this interfere with the former setup?
do you want me to delete this folder with the files from the " Create AI Images" and start over with a new folder now? Wouldn't this interfere with the former setup?
Date: 05/12/2022 08:24
User: Jorge Vila
Awards:
User: Jorge Vila
Awards:
No, you must delete SD folder everytime, the proccess is not really the same, so try it and let mw know!
I'm here to help!
I'm here to help!
Date: 05/12/2022 11:20
User: Neves Chrichlow
Awards:
User: Neves Chrichlow
Awards:
I deleted the SD folder since last night and was trying. Attempts to get the processor was delayed and still is. I am getting the following alert
" Cannot connect to GPU backend
You cannot currently connect to a GPU due to usage limits in Colab.
Learn more"
" Cannot connect to GPU backend
You cannot currently connect to a GPU due to usage limits in Colab.
Learn more"
Date: 09/12/2022 10:22
User: Jorge Vila
Awards:
User: Jorge Vila
Awards:
Respuesta Aprobada
OK, this happens sometimes when the Google Colab is saturated, I recommend to try in an hour to get a free GPU!
Otherwise, contact me throgh jorge@jorgevila.com
Otherwise, contact me throgh jorge@jorgevila.com