|
Sphinx
Community
Services
Misc
 Subscribe in a reader
|
trouble while reindexing
Common forum |
1 | 2 | 3 | 4 | 5 | ... |
224 | 225 | 226 | 227 | next »» | Create new thread
|
amauf
Name: Andreas Mauf
Posts: 17
|
2007-01-27 13:54:57
| reply!
Hi,
searchd is getting in trouble while sorting after the docs are collected. The PHP-Api
triggers the following errors:
Warning: unpack(): Type n: not enough input, need 2, have 0 in sphinxapi.php on line 160
Warning: array_values(): The argument should be an array in sphinxapi.php on line 160
I also noticed that while sorting all searches don't give any results back. Trying to
restart the searchd doesn't work:
# kill 11618
# ../bin/searchd
Sphinx 0.9.7-RC2
Copyright (c) 2001-2006, Andrew Aksyonoff
using config file 'sphinx.conf'...
WARNING: index 'abc': failed to preload schema and docinfos - NOT SERVING
FATAL: no valid indexes to serve
In searchd.log there are a lot of errors:
WARNING: maxed out, dismissing client
After sorting is finished and starting searchd again, all works fine as before reindexing.
What was happen? I thougt the indexer using tmp-files while indexing (there where 2 temp
files), so that the old index is still usable. Does the indexer automatically restarts
the searchd after indexing?
Thanks.
|
|
|
psdopsdo
Name: psdopsdo
Posts: 14
|
to: amauf, 2007-01-28 04:50:31
| reply!
> Hi,
>
> searchd is getting in trouble while sorting after the docs are collected. The PHP-Api
> triggers the following errors:
>
> Warning: unpack(): Type n: not enough input, need 2, have 0 in sphinxapi.php on line 160
> Warning: array_values(): The argument should be an array in sphinxapi.php on line 160
>
> I also noticed that while sorting all searches don't give any results back. Trying to
> restart the searchd doesn't work:
>
> # kill 11618
> # ../bin/searchd
> Sphinx 0.9.7-RC2
> Copyright (c) 2001-2006, Andrew Aksyonoff
>
> using config file 'sphinx.conf'...
> WARNING: index 'abc': failed to preload schema and docinfos - NOT SERVING
> FATAL: no valid indexes to serve
>
> In searchd.log there are a lot of errors:
> WARNING: maxed out, dismissing client
>
> After sorting is finished and starting searchd again, all works fine as before reindexing.
>
> What was happen? I thougt the indexer using tmp-files while indexing (there where 2 temp
> files), so that the old index is still usable. Does the indexer automatically restarts
> the searchd after indexing?
>
> Thanks.
>
Hi,
I don't know the reason of the message :
"Warning: unpack(): Type n: not enough input, need 2, have 0 in sphinxapi.php on line 160
..."
I had it when i tried to have index="*" or index="myindex1 myindex2" with sphinxapi.php.
I resolved it by using a loop (the php syntax below is incorrect)
for i = myindex1, myindex2 ...
cl->query($i);
done
Take a look at : http://www.sphinxsearch.com/forum/view.html?id=245
|
|
|
shodan
Name: Andrew Aksyonoff
Posts: 3966
|
to: amauf, 2007-01-28 18:23:55
| reply!
> What was happen? I thougt the indexer using tmp-files while indexing (there where 2 temp
> files), so that the old index is still usable.
It seems that
a) the lock files were lost somehow, and
b) you ran indexer without --rotate switch.
Normally, searchd would create .spl lock files on startup (and kill them on shutdown),
and indexer would then refuse to reindex without --rotate switch.
> Does the indexer automatically restarts the searchd after indexing?
If used with --rotate.
|
|
shodan
Name: Andrew Aksyonoff
Posts: 3966
|
to: psdopsdo, 2007-01-28 18:24:44
| reply!
> I don't know the reason of the message :
> "Warning: unpack(): Type n: not enough input, need 2, have 0 in sphinxapi.php on line 160
> ..."
The API would also return an error about "zero-sized searchd response" in this case. It
usually means that searchd crashed for some reason :(
|
|
|
amauf
Name: Andreas Mauf
Posts: 17
|
to: shodan, 2007-01-30 11:45:56
| reply!
> > Does the indexer automatically restarts the searchd after indexing?
>
> If used with --rotate.
With the "--rotate" switch it works. I've missed that in the documentation. ;-)
Now I have some new files in the data directory. I'm not sure which ones searchd is
using, still the old ones?
# ../bin/indexer --rotate abc
Sphinx 0.9.7-RC2
Copyright (c) 2001-2006, Andrew Aksyonoff
using config file 'sphinx.conf'...
indexing index 'abc'...
collected 9642641 docs, 2817.0 MB
sorted 416.5 Mhits, 100.0% done
total 9642641 docs, 2817025336 bytes
total 4814.913 sec, 585062.61 bytes/sec, 2002.66 docs/sec
rotating indices: succesfully sent SIGHUP to searchd (pid=22175).
# ls -ahl /home/abc/sphinx/
total 7.0G
drwxr-xr-x 2 abc abc 1.0K Jan 30 14:36 .
drwxr-xr-x 3 abc abc 1.0K Jan 23 17:19 ..
-rw-r--r-- 1 root root 184M Jan 30 14:19 sphinx.new.spa
-rw-r--r-- 1 root root 3.2G Jan 30 14:36 sphinx.new.spd
-rw-r--r-- 1 root root 151 Jan 30 14:36 sphinx.new.sph
-rw-r--r-- 1 root root 1.8M Jan 30 14:36 sphinx.new.spi
-rw-r--r-- 1 root root 197M Jan 27 19:24 sphinx.spa
-rw-r--r-- 1 root root 3.5G Jan 27 19:41 sphinx.spd
-rw-r--r-- 1 root root 151 Jan 27 19:41 sphinx.sph
-rw-r--r-- 1 root root 2.0M Jan 27 19:41 sphinx.spi
|
|
|
shodan
Name: Andrew Aksyonoff
Posts: 3966
|
to: amauf, 2007-01-30 17:21:34
| reply!
> Now I have some new files in the data directory. I'm not sure which ones searchd is
> using, still the old ones?
Still the old ones; it would had renamed current to .old on sucessful rotation.
Check searchd.log if there were any errors - for instance if it was running under
non-root account it might had not be able to pick up the new ones.
|
|
|
amauf
Name: Andreas Mauf
Posts: 17
|
to: shodan, 2007-01-30 18:00:32
| reply!
> Still the old ones; it would had renamed current to .old on sucessful rotation.
>
> Check searchd.log if there were any errors - for instance if it was running under
> non-root account it might had not be able to pick up the new ones.
I forgot about it... Running under root.
[Tue Jan 30 14:36:16 2007] [22175] rotating indices: caught SIGHUP, waiting for
children to exit
[Tue Jan 30 14:36:16 2007] [22175] rotating index 'abc': children exited, tryi
ng to rotate
[Tue Jan 30 14:36:16 2007] [22175] WARNING: rotating index 'abc': failed to pr
eload schema/docinfo
[Tue Jan 30 14:36:16 2007] [22175] WARNING: rotating index 'abc': using old in
dex
|
|
|
shodan
Name: Andrew Aksyonoff
Posts: 3966
|
to: amauf, 2007-01-30 18:33:09
| reply!
> [Tue Jan 30 14:36:16 2007] [22175] WARNING: rotating index 'abc': failed to pr
> eload schema/docinfo
In other words searchd failed to load new .sph and/or .spa files for some reason and
rolled back to old index.
|
|
amauf
Name: Andreas Mauf
Posts: 17
|
to: shodan, 2007-01-31 10:44:09
| reply!
> > [Tue Jan 30 14:36:16 2007] [22175] WARNING: rotating index 'abc': failed to pr
> > eload schema/docinfo
>
> In other words searchd failed to load new .sph and/or .spa files for some reason and
> rolled back to old index.
Next run, same error again. What happens when rotating in detail? If I manually stop
searchd, delete the old files, rename the new ones to the old name and start searchd it
works.
|
|
|
shodan
Name: Andrew Aksyonoff
Posts: 3966
|
to: amauf, 2007-02-03 08:59:36
| reply!
> Next run, same error again. What happens when rotating in detail?
indexers sends SIGHUP to searchd when completed (and it should print that out). searchd
then does the following:
1) waits for all children to exit
2) renames current index files to .old
3) renames .new index files to current
4) tries to load current index files
5) if something failed, it rollbacks current files from .old and new to .new
In your case I would start with double checking directory access rights, users which both
indexer and searchd run under, etc.
|
|
shodan
Name: Andrew Aksyonoff
Posts: 3966
|
to: amauf, 2007-02-03 09:02:44
| reply!
> [Tue Jan 30 14:36:16 2007] [22175] WARNING: rotating index 'abc': failed to pr
> eload schema/docinfo
This might happen for a number of reasons some of which could not be easily diagnosed
now. I think I'll fix the code to report the exact reason and update you with a fresh
snapshot.
|
|
amauf
Name: Andreas Mauf
Posts: 17
|
to: shodan, 2007-02-03 12:02:40
| reply!
> I think I'll fix the code to report the exact reason and update you with a fresh snapshot.
This would be fine. File permission should be ok. I think its a memory problem, cause
I've found some alerts in the "resource log". The total size of shared memory is limited
on the current machine. About 80% are used in regular, if searchd is running. Could it be
that the old data are not freed before the new data is loaded, or something like that?
|
|
shodan
Name: Andrew Aksyonoff
Posts: 3966
|
to: amauf, 2007-02-05 02:25:00
| reply!
> Could it be that the old data are not freed before the new data is loaded, or something
> like that?
Yes, this seems possible.
|
|
amauf
Name: Andreas Mauf
Posts: 17
|
to: shodan, 2007-03-06 07:03:28
| reply!
I have two indexes now and reindex them with "--rotate --all".
But the indexer does rotate only one index, the second is left in 'new'-state. The
searchd.log talks only about first index, which is successfull rotated. No hint about the
other index.
|
|
shodan
Name: Andrew Aksyonoff
Posts: 3966
|
to: amauf, 2007-03-07 18:49:48
| reply!
> The searchd.log talks only about first index, which is successfull rotated.
I suppose that you added the second one but did not restart searchd since.
At the moment it is only able to pick up new index data on SIGHUP, but not new index
definitions from sphinx.conf
Restarting searchd should help.
|
Common forum |
1 | 2 | 3 | 4 | 5 | ... |
224 | 225 | 226 | 227 | next »» | Create new thread
|
