themaLeecher
http://leecher.themasoftware.com/forum/

Regex Search For Duplicates
http://leecher.themasoftware.com/forum/viewtopic.php?f=4&t=6711		 Page 1 of 1
Author:  Pablo01 [ May 19th, 2023, 2:19 pm ]
Post subject:  Regex Search For Duplicates
Hi,

I am wondering if there is anyway to use search to find duplicate posts by certain string in Subject title.

Example:

Code:
|40:54| MySecretLifePOV FREE Page - 2022.09.08 - @mysecretlifepov @katalinakyle ID: 2584453353
|40:54| MySecretLifePOV FREE Page - 2022.09.08 - @mysecretlifepov @katalinakyleu00a0and ID: 2584453353


Code:
|41:15| MySecretLifePOV FREE Page - 2022.11.14 - @mysecretlifepov @mollymaeroyvip @badbellaxo ID: 2669179277
|41:15| MySecretLifePOV FREE Page - 2022.11.14 - @mysecretlifepov @mollymaeroyvip @badbellaxou00a0both ID: 2669179277
|41:15| MySecretLifePOV FREE Page - 2022.11.14 - @mysecretlifepov @mollymaeroyvipu00a0came @badbellaxou00a0both ID: 2669179277


As you can see the part ID: xxxxxxxx are the same but I would like to use the search in TL to find ALL posts that have the same "ID: xxxxx" part - to sort out duplicates.

Now I do it manually by sorting the posts and looking at the running time to the left and then check if the ID:xxxxx is the same - very tedious work :(

Is there a regex or other method to find it quickly ?
Author:  Freddy [ May 21st, 2023, 5:38 am ]
Post subject:  Re: Regex Search For Duplicates
Hi,

no, I can't think of way to do that more easily in themaLeecher.

That way too specific, it would need a custom script to filter like that.

The only way I can think is to save all posts to .txt files -> use some custom script to filter and delete not needed -> delete all those messages from themaLeecher -> then load back what's left of .txt files to themaLeecher if needed.
Author:  Freddy [ May 21st, 2023, 5:43 am ]
Post subject:  Re: Regex Search For Duplicates
Well, I might actually add a new option to "Match search on duplicates only".

Then it might be possible with regex.

Will take a look more.
Author:  Pablo01 [ May 21st, 2023, 7:01 am ]
Post subject:  Re: Regex Search For Duplicates
Thanks for checking.

The thing here is - I want to keep the duplicates in themaLeecher.
Simply because some creators re-post their content but the videos have same ID - this way I can identify if its a repost.
Also when they re-post - sometimes they delete the old messages or the old messages.
Anyway - I want to keep the posts with same ID

I just need a way to find all posts who have the same ID.

I had my custom RSS changed now so ID NUmber is Second in Subject and runtime is first - then I can sort A-Z but it is still manual checking.

This is how the subject will look:
Code:
 
|32:45| [287164416] Don sudan - 2023.05.16 - bla bla
|33:33| [2828957843] Don sudan - 2023.04.16 - txt xtt
|33:33| [2828957843]Don sudan - 2023.04.16 - dumdidum
|23:33| [14545455] Don sudan - 2023.04.16 - text text

so basically there is now 3 ID´s and I would need a way to filter out to only see the ones with Identical ID in the brackets.

I guess I just hoped there was a regex search string to find duplicate text - imagine 5000 posts or more where I have to look manually :)
Author:  Freddy [ May 21st, 2023, 2:25 pm ]
Post subject:  Re: Regex Search For Duplicates
Will try to add a separate option for that.
Author:  Freddy [ May 27th, 2023, 11:56 am ]
Post subject:  Re: Regex Search For Duplicates
4.72:
* Added new search option for messages: "Search text regex matching only duplicates".

For you last example use this regex with that option enable and it will find what you need:
Code:
\[[0-9]+\]
Author:  Pablo01 [ May 29th, 2023, 2:14 pm ]
Post subject:  Re: Regex Search For Duplicates
Thank you! this is really good.

one last regex question regarding this.

I want to find any string of numbers that are duplicates in subject.

Can be [12345] or 12345 or _12345_

is there a solution for that so that its not limited to duplicates withing brackets ?
Author:  Freddy [ May 29th, 2023, 4:31 pm ]
Post subject:  Re: Regex Search For Duplicates
Try this:

Code:
.[0-9]+.
Author:  Pablo01 [ May 29th, 2023, 5:03 pm ]
Post subject:  Re: Regex Search For Duplicates
thanks, that did not work for me - it finds subjects that dont have a duplicate string of numbers.
Author:  Freddy [ May 29th, 2023, 7:49 pm ]
Post subject:  Re: Regex Search For Duplicates
I would need some messages to test on.

If you can do this: "Help" menu (at the top) -> "Support" -> "Compress" -> send me the file which it opens (via PM in any file host).

Will review your settings.
Page 1 of 1 All times are UTC
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/